{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "This example demonstrates how to convert a network from [Caffe's Model Zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo) for use with Lasagne. We will be using the Caffe version of CIFAR-10 Model.\n",
    "\n",
    "We will create a set of Lasagne layers corresponding to the Caffe model specification (prototxt), then copy the parameters from the caffemodel file into our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Converting from Caffe to Lasagne\n",
    "### Download the required files\n",
    "\n",
    "First we download `cifar10_nin.caffemodel` and `model.prototxt`. The supplied `train_val.prototxt` was modified to replace the data layers with an input specification, and remove the unneeded loss/accuracy layers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Caffe\n",
    "\n",
    "To load the saved parameters, we'll need to have Caffe's Python bindings installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "caffe_root = '/home/xilinx/caffe/'  # this file should be run from {caffe_root}/examples (otherwise change this line)\n",
    "sys.path.insert(0, caffe_root + 'python')\n",
    "import caffe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the pretrained Caffe network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net_caffe = caffe.Net('CIFAR_10.prototxt', 'cifar10_quick_iter_5000.caffemodel.h5', caffe.TEST)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import lasagne\n",
    "from lasagne.layers import InputLayer, DropoutLayer, DenseLayer, NonlinearityLayer\n",
    "#from lasagne.layers.dnn import Conv2DDNNLayer as ConvLayer\n",
    "from lasagne.layers import Conv2DLayer as ConvLayer\n",
    "from lasagne.layers import Pool2DLayer as PoolLayer\n",
    "from lasagne.nonlinearities import softmax, rectify, linear\n",
    "import conv_fpga\n",
    "from conv_fpga import FPGA_CIFAR10\n",
    "from conv_fpga import FPGAQuickTest\n",
    "#from conv_fpga import Conv2DLayer as ConvLayer\n",
    "from conv_fpga import FPGAWeightLoader as FPGALoadW\n",
    "from lasagne.utils import floatX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a Lasagne network\n",
    "Layer names match those in `model.prototxt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net = {}\n",
    "net['input'] = InputLayer((None, 3, 32, 32))\n",
    "net['conv1'] = ConvLayer(net['input'], num_filters=32, filter_size=5, pad=2, nonlinearity=None)\n",
    "net['pool1'] = PoolLayer(net['conv1'], pool_size=2, stride=2, mode='max', ignore_border=False)\n",
    "net['relu1'] = NonlinearityLayer(net['pool1'], rectify)\n",
    "net['conv2'] = ConvLayer(net['relu1'], num_filters=32, filter_size=5, pad=2, nonlinearity=rectify)\n",
    "net['pool2'] = PoolLayer(net['conv2'], pool_size=2, stride=2, mode='average_exc_pad', ignore_border=False)\n",
    "net['conv3'] = ConvLayer(net['pool2'], num_filters=64, filter_size=5, pad=2, nonlinearity=rectify)\n",
    "net['pool3'] = PoolLayer(net['conv3'], pool_size=2, stride=2, mode='average_exc_pad', ignore_border=False)\n",
    "net['ip1'] = DenseLayer(net['pool3'], num_units=64, nonlinearity = None)\n",
    "net['ip2'] = DenseLayer(net['ip1'], num_units=10, nonlinearity = None)\n",
    "net['prob'] = NonlinearityLayer(net['ip2'], softmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Copy the parameters from Caffe to Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "layers_caffe = dict(zip(list(net_caffe._layer_names), net_caffe.layers))\n",
    "\n",
    "for name, layer in net.items():\n",
    "    try:\n",
    "#        layer.W.set_value(layers_caffe[name].blobs[0].data)\n",
    "#        layer.b.set_value(layers_caffe[name].blobs[1].data)     \n",
    "        if name=='ip1'or name=='ip2':\n",
    "            layer.W.set_value(np.transpose(layers_caffe[name].blobs[0].data))\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "#             print((layers_caffe[name].blobs[0].data.shape))\n",
    "#             print(np.amax(layers_caffe[name].blobs[0].data))\n",
    "#             print(np.amin(layers_caffe[name].blobs[0].data))\n",
    "        else:\n",
    "            layer.W.set_value(layers_caffe[name].blobs[0].data[:,:,::-1,::-1])\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "#             print((layers_caffe[name].blobs[0].data.shape))\n",
    "#             print(np.amax(layers_caffe[name].blobs[0].data))\n",
    "#             print(np.amin(layers_caffe[name].blobs[0].data))\n",
    "            \n",
    "    except AttributeError:\n",
    "        continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Copy the parameters from CPU to FPGA OnChip Memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight shape (32, 3, 5, 5)\n",
      "kermax 0.191131100059\n",
      "kermin -0.26925727725\n",
      "Elapsed Test Time:  0.0028484150000025465\n",
      "weight shape (32, 32, 5, 5)\n",
      "kermax 0.112733885646\n",
      "kermin -0.121608451009\n",
      "Elapsed Test Time:  0.000710628000000213\n",
      "weight shape (64, 32, 5, 5)\n",
      "kermax 0.0932634323835\n",
      "kermin -0.0825128629804\n",
      "Elapsed Test Time:  0.0012873229999996738\n",
      "weight shape (64, 64, 4, 4)\n",
      "kermax 0.366065233946\n",
      "kermin -0.347633630037\n",
      "Elapsed Test Time:  0.0016130619999970008\n",
      "weight shape (10, 64, 1, 1)\n",
      "kermax 0.284343063831\n",
      "kermin -0.279969036579\n",
      "Elapsed Test Time:  0.00020939399999875263\n"
     ]
    }
   ],
   "source": [
    "#FPGALoadW(weight, status, IFDim, OFDim, PadDim)\n",
    "weight = net['conv1'].W.get_value()\n",
    "FPGALoadW(weight, 1, 32, 32, 2)\n",
    "weight = net['conv2'].W.get_value()\n",
    "FPGALoadW(weight, 2, 16, 16, 2)\n",
    "weight = net['conv3'].W.get_value()\n",
    "FPGALoadW(weight, 3, 8, 8, 2)\n",
    "weight = net['ip1'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(64, 64, 4, 4)\n",
    "FPGALoadW(weight, 4, 4, 1, 0, flip_filters=False)\n",
    "weight = net['ip2'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(10, 64, 1, 1)\n",
    "FPGALoadW(weight, 5, 1, 1, 0, flip_filters=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Trying it out\n",
    "Let's see if that worked. \n",
    "\n",
    "### Import numpy and set up plotting\n",
    "### Import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gzip\n",
    "import _pickle as cPickle\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download some test data\n",
    "Load CIFAR_10 test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x2a16d3d0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATYAAAE0CAYAAACmdB+iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3WtwnNWZJ/D/0zfd75Yl2TLIF2wuITGQEFhIDQy5sFNs\nyGRrsxlmqhKmKpWtmsymKvshly/UfNmaTG1Rlb2kMpsQilAwGSazGUhmikuGzWxggjFgExtjG19k\nLFuSJVl39b3PflDbyEZ9/i+W3GoO/18VhdTn8enTb7969Hbr6eeYcw4iIiGJrfUCRERWmxKbiARH\niU1EgqPEJiLBUWITkeAosYlIcFaU2MzsbjM7aGaHzeybq7UoEZGVsEutYzOzGIDDAO4CcBrAbgBf\ndM4dvChOhXIiclk452y52xMrmPNmAG85504AgJn9FMC9AA5eHPiD/3TV+a9/sXsC/+5jXReMmyt5\n76gumaSLsRi/+MzmMjSmUMxf8P0vX53GPTe1vbOWVIrPUfI/HgBwJZ7vLVb0jsfiF37/1K5pfPbj\nbRfc5vLN/H5QoDGplP/YxSOcSha78DH//UuT+Pe3dFxwW7HE15Iv+I9vsbTsuX7RYvh6C8UL5/nl\nq5O456YL15sl9xVhJShFOF8sxmfKZt85d597fQaf+kjrBePFYoTniPwsAkAM/vMSAHLkMc1f9DS/\ncGAWt1/bcmFM1n8///MfxyqOreSl6EYAJ5d8P1S+TURkTemPByISnJW8FD0F4Iol3/eXb3uXX+ye\nOP91Q+r9lUu399Wt9RIi27Hx/bNWALimv36tl/CebO97/6x3S8/761y4opu/xTM0kcOpiTyNA1b2\nx4M4gENY/OPBMICXAfyRc+7Ni+Lc0vfYll9E7b7H9q611PB7bMveTw2/x7acWn6PbTm1+h7bcmr5\nPbZlYyK8x7bqfzxwzhXN7GsAnsXiS9qHLk5qIiJrYSUvReGcexrAjlVai4jIqlhRYosqB/9rJufS\n/gkiXKrXg7/kipN1AAAS5OVfhLcIExFe3VuST5TJ5bzjhRJ/PAnH7ycR56dBgkxjpQjvfRSyNCQW\n4WVxseRfb874e2HFOH8PKkfuBwCyBf+BsRJ/2Rbl2DVEOF+S5o+JRzgxC/kIz6Px58iR59FF+Ltl\nnJ10Hu+vd/JFRCJQYhOR4CixiUhwlNhEJDhKbCISHCU2EQmOEpuIBEeJTUSCU5UCXccKEJ2/cNMV\neUGgFXmxapEUvAJAopEUXIIXC8cj1AGXIhRuss/I5h3/DG0pH+G4RFhLoeCPsQifOY5FKBa2eCON\ncXF/AW66yAt0T4/zc2E+xx/T7Kx/nrjjx7a1nj9HKePnXVuT/9g11PGfo1KMH5dYhE/AxknRNz9z\ngVyEz1NXois2EQmOEpuIBEeJTUSCo8QmIsFRYhOR4CixiUhwlNhEJDhVqWNLFEmDwbi/XiVW4rU1\ndfEITdSTETrQk06SsXiE3wURym8KUWp0SJ/7ZIrXfPVtuZrGTE9W3p/xnLHxee94KsH3goiB15dl\nC/yUTLsG7/j+4/zxuLouGpNP8OaluRb/Y5qdmvCOA8DQ6CSNaa7nlV/FYf88A738OVrXyhtw1iei\n7J3g/3lMRfhRLEaoAaxEV2wiEhwlNhEJjhKbiARHiU1EgqPEJiLBUWITkeAosYlIcJTYRCQ4VSnQ\nBWlMZ4l2/7jxar6C4434YjFexJsr+IuBUxF2EC8WeWGhi9DcEeRxp1L899K/+cy/pTEv/+YFGnNq\nctw7Ph+hsDZf5AWvg2+P0pijQye943UdG+kcV/RupTGuvpXGZBP+8yHZvJ7OUcjM0Zjx0SEa09jh\nLzp+e26YzpEu8Z+jvlZeLNyY9DfPLOYX6ByxS+8zqSs2EQmPEpuIBEeJTUSCo8QmIsFRYhOR4Cix\niUhwlNhEJDhKbCISnBUV6JrZIIBpACUAeefczcvFZWP+QsepeX8n2GKBdOAF0NnCi29b47woNkl2\nNC+RAl4AsAiFha7E18u69S7Mn6VzPPsPP6MxI5P8+I7M+tdyfIiv5fipEzQm0dBCYwrxNu94c2s3\nnSPZxO8nUe/v1AsAdeY/Lg0xXpQ8lkvTmA2bBmhMJu0v9D16lBfoTkxlaEzC+GPavN5/fJNFXghs\nxQhdsStY6ScPSgDucM7x3sYiIlWy0peitgpziIisqpUmJQfgOTPbbWZfWY0FiYis1Epfit7mnBs2\ns24sJrg3nXP8E9UiIpfRihKbc264/P8xM/s5gJsBvCuxPf3qmfNfb+trwrYNTSu5WxH5ABo8k8Hg\nGf6HLmAFic3MGgHEnHNzZtYE4NMA/mK52Ltv4q1bRER8BtbXY2D9O/u4/suB2YqxK7li6wHwczNz\n5Xkec849u4L5RERWxSUnNufccQA7V3EtIiKroioddM8s+LtpTuQ7vOPP/+Z5eh8f2sHft/vk9bxw\nszFOCnQjdMeNxf2PFwBiMd6FtOjy3nFSGwoAODZ4jMZMLPCuwK6p0zseb+YFr7HOGRrT0OHvpgwA\nuYy/oDVrvPizrZOfL23NvBB1ZNhf9Dp9doLO0ZriP4b1jbxYePCs/7gkW3vpHGdOD9KYEyOVXwKe\n09fmL7pvMP6YCyX/+e+jGjQRCY4Sm4gER4lNRIKjxCYiwVFiE5HgKLGJSHCU2EQkOFWpY0u0+3fd\nnh/359d8in8ka2Ke147N53gtUFvK30iy5CI0vyvxTpPxuL/OBwDSOX9MlI/Njc3w9TZ1bKAxnesH\nvONzxWk6x3rw4x9v4Mclm/LHZOZ4nVV6ltfUbe5dR2PmSQ3aaI7veG7JehozNTFPY1Dy11im5/iO\n8/E6fvxHp3n7xdNT/pq6gW6eemK8HLHyv730fyoiUpuU2EQkOEpsIhIcJTYRCY4Sm4gER4lNRIKj\nxCYiwVFiE5HgVKVA95qdt3rH337xgHe8pZ0X6N5yu/8+AKApfpzGZOf9hZvxBG8QaUleiFp0/uaa\nANDac6V3/LW9h+kcLe28yLR/4Hoa42L+ItJkilcLF7PjNCab40WkMfIcxCM0Mdy3Zw+Naavnz3VT\ns79hZXMjb8A5NDxCYwpRir6T/oahna38vJya5M0dz074i9gB4Oiwv2B7Y28fnSMR4ZyqRFdsIhIc\nJTYRCY4Sm4gER4lNRIKjxCYiwVFiE5HgKLGJSHCU2EQkOFUp0G0kRaKbt13tHU9H2BB6YOt2GtOd\n50WOk0cHveN5sjs7ABQLfJfxW+/8DzRmYNvN3vGtH+EFxy+/+hqN6WzhHXRPjY55x5OO7yZfn+QF\nrxn+FGFu3t9NdirC7uudTXwtEZZCC2e71/Pi8kyedzkeOztFYyzuv05pibCzfSLOj0s2zbv5Hjlx\n0ju+voMXC+/Y1EpjKtEVm4gER4lNRIKjxCYiwVFiE5HgKLGJSHCU2EQkOEpsIhIcmtjM7CEzGzWz\n3y25rcPMnjWzQ2b2jJm1Xd5liohEF6VA92EA/wPAT5bc9i0Av3LO/ZWZfRPAt8u3LX8n9f7CwFMj\n+73jN958C11kczsvio3P+IsGAaBYKHnHEylewHjkbX8XXgD4vc4tNAZNV3iHW5sX6BQNCd7BtTHV\nSGPqU6QAt8SLTPv7eSHw/reO0JhUyt/Nd3qGH/8tm3bQmKuv/RCNmZg46x1vaTM6x6nhURpjsTiN\n6ejs8o5Pz/jXCgA5UuQLAI1NvPtzetZ/bh4+wZ+jhtSlv6Ck/9I59wKAyYtuvhfAI+WvHwHwuUte\ngYjIKrvUlLjeOTcKAM65EQD8cyMiIlWyWn88iPKxOhGRqrjUD8GPmlmPc27UzHoBnPEFP/74E+e/\nvv7663D99ddd4t2KyAfVyNk0Rs6mI8VGTWxW/u+cpwB8GcB3AXwJwJO+f3zffV+IeDciIsvr7WxA\nb+c7XUFeP1q540mUco/HAfwrgO1m9raZ3Q/gLwF8yswOAbir/L2ISE2gV2zOufsqDH1yldciIrIq\nqtJoMlnvr99NZ/w7S2ezvLljMkItVlMzryNurvfPUxfn9VotiQyN+eH/+gGN+cM/+YZ3PDk3TOdI\n1fG/D8Vi/DFt3dbvHR+dGKJzpOd4g8K+nm4aMzHtnyeb4zuVb9vOG5Netd3fABUApl7Z7R2fn+E7\n20+TxpkAUCj66ysBIJ32n3ft7e38fkq8vqytg9dyFnL+cyoe4z8jJ0/z+r5K9JEqEQmOEpuIBEeJ\nTUSCo8QmIsFRYhOR4CixiUhwlNhEJDhKbCISnKoU6BrZXXph1l/EmJnnH3xNJv3NBwFgZrxIYxD3\n71CdxDSdYkMHbwp4aP9hGnPqbRKzcIrOcfzkMRrz0Q230pj+zf4mkRtH++gcc4f5zvVddbyItLXD\nX8R75Ah/zBv6/QXHADA5zZ/rPCmcHT4zTucoOd6M0uL8R3V+wf9zYhEKsflKgKYIO8qj5G96mTLe\nJDU3zgvQK9EVm4gER4lNRIKjxCYiwVFiE5HgKLGJSHCU2EQkOEpsIhIcJTYRCU5VCnRR8u/OF3f+\nIscN3evoXTQ1kJ3KATy79y0a05n3r2VHF+8e2lDHCyFTSd5B9Myov9C0lOU7ew9ctZXGxOv5sWtq\n7fSOd/duonOMT8zSmOlpXrhZJHXWPT18m9tEhILuDOkCCwC5vD8mneHPc77AC8cL7EEDyGT9nYML\nBX4ds667h8aY8Z+Bupi/WLjO+LEtuiYSUbnzsK7YRCQ4SmwiEhwlNhEJjhKbiARHiU1EgqPEJiLB\nUWITkeAosYlIcKpSoJtM+DvKtrc0esc7Wv1dbQHASrzgb9rxzp9jk/4eot2t/JA1p1I0phibojHH\nhvwdZ3s72+gcA9s/RGMy/rpOAMBvd+/3jg+d4sXCrc3+Il8ASCX5c/27w4MkgncwLkX4nZ7J8nNq\nds5fiNrR5e8kCwCFCB10Tw2P0pjmVn/34UTcXygPAE1N/p9FAEileHEz8v7OwcW5STpFb08LiThT\ncURXbCISHCU2EQmOEpuIBEeJTUSCo8QmIsFRYhOR4CixiUhwaFGWmT0E4B4Ao865D5dvewDAV/BO\nIcl3nHNPV5ojbv46nb4e/y7iiQj5t5TO0piNm7bQmF1D/uaOk8Zr4VzCv7M9ALR388aB7W3+hn7J\nhlY6x5YIdWwtbbyR5//+6x97x+cz/PhPL0zQmPk0P3Ypctb2dfJGiJkJviv9fF2U58jfDPHAm0N0\njuHhyvVY58zM8iad7WS3+NYmVhcGxF2exiRzlRs8np9n/pR3fH0zL55sr4+yL/3yolyxPQzgM8vc\n/qBz7sbyfxWTmohItdHE5px7AcByZcKXnk5FRC6jlbzH9jUz22tmPzIz/tkeEZEqudTE9n0AW5xz\nOwGMAHhw9ZYkIrIyl/QheOfc2JJvfwjgF774hx9++PzXO3fuxA033HApdysiH2DHRxYwOOJvOnBO\n1MRmWPKempn1OudGyt9+HoC39cP9998f8W5ERJa3ubcRm3vf6T7y69crdwiJUu7xOIA7AHSZ2dsA\nHgBwp5ntBFACMAjgqytasYjIKqKJzTl33zI3P7zMbSIiNaEqjSbr6vyN6do6/QW6hSJfZl2S72Z+\n9dYBGrPrZX/R60xqO52jZDM0prefF5Huf+NF7/gnfv8rdI4XfuOfAwDm56ZpTC435h0fPf02nSPK\n36pm8zwmAX9xZ2eMN73sb+CPeerMIRpTiPubZ/b28OaaxQJvaJlO8x3lMwsL3vH5JG8QWYhQCJxP\n86Lj3pR/LRtb2C7vQDbvn8NHH6kSkeAosYlIcJTYRCQ4SmwiEhwlNhEJjhKbiARHiU1EgqPEJiLB\nqUqBblOzvxivs7vbO16wCB1RY3z39foW3l2po8MfM3jiNJ3j9275MI3JzJZoTGOrv7Pq6SFeFHv4\n4EEaUyjybqYxsrn63Azf2b513QYaMzXNizLbm/27xW+5mh//l/YcoDG7D/Auu3fcdY93PFnHC1GP\nvHWYxkzN8OPCdrfPpHnx7UAv78rcQI4/AHR2+bv1ugQvSs7H+M71leiKTUSCo8QmIsFRYhOR4Cix\niUhwlNhEJDhKbCISHCU2EQmOEpuIBKcqBbqlgr+4sI0U880vFOl9zBd5MV88zvP4lVde4R0/uI93\nVZ2a5+ttab6Sr+Uq//jxg7yAdGiIFxTffvstNGZ+wV/c2bqxn87RtXErjRmc4IWz6ay/uDnV1EXn\naFvPj/9HW/3nAgCcOTPuHT92fA+dYy7NC6Qnp3hx7fr1673jbY6fC5ub+Vp62ki1NoAk6SKdi9Ad\nt9kufU92XbGJSHCU2EQkOEpsIhIcJTYRCY4Sm4gER4lNRIKjxCYiwalKHdvM+CnveCPZoTprvLbG\nIuwWb8Zr3dZ3rfOOH4wdoXOMTszTmPEp3miyvaXPO37dh9vpHEeP82aUeV52h6lp/2PasWMHnWPH\nVlKYB+D4ad6wct++173j42ONdI5UXTON6WzhTRdPDvvr7obH+Y7zFudNUuP1fC0br/DXCQ5EKAsb\naOW7xdfHeJPIbMZ/fpdKvHlsvsDvpxJdsYlIcJTYRCQ4SmwiEhwlNhEJjhKbiARHiU1EgqPEJiLB\nUWITkeDQqlYz6wfwEwA9AEoAfuic++9m1gHgbwFcCWAQwBecc8tWIx49fNR7H1fuuM47Xh/LsmWi\nlOON65LxCMWH9f6Y1hZ/U0wAaGnjO85fe+01NObpf3rKO74wzRsHNnb10JjDJ0dozBWbBrzjW675\nGJ2jro4XUW8b8N8PAExOnPWO73+DNwMtOl6VPDTJC8On0/55MkV+zk1P8YLunj7e9HJw3D9P1xW8\noHu8vo7GoBihMWbBf1xcgu8mnymyn/tjFUeiXLEVAHzDOXcdgFsB/JmZXQ3gWwB+5ZzbAeB5AN+O\nMJeIyGVHE5tzbsQ5t7f89RyANwH0A7gXwCPlsEcAfO5yLVJE5L14T++xmdkAgJ0AXgLQ45wbBRaT\nHwB/w3URkSqJnNjMrBnAzwB8vXzldvEnyvknzEVEqiBSdw8zS2AxqT3qnHuyfPOomfU450bNrBfA\nmUr//ue/fOb811dv34prtm9bwZJF5IPo5PAMhkb4bl1A9LZFPwZwwDn3vSW3PQXgywC+C+BLAJ5c\n5t8BAP7wns9EvBsRkeVt6mvFpr532jften24YmyUco/bAPwxgH1mtgeLLzm/g8WE9oSZ/SmAEwC+\nsLJli4isDprYnHMvAqi0Q+onV3c5IiIrV5UOuq8e9heAXvmRW73jJczR+7Ao3TYj7BY/Pet/DT85\nOUbnWNd1I4357D38d8JNN17rHX/sb/+OzmHGd+1ub++kMf39m7zjLa0ddI54gT+PXX28s+rGrf4C\n0alGXvz5ymuv0ZjTs7zlrEv5O9u2bfB3ZAaA7iZeOBuPUNBadP71vuma6ByHT/PC5VScH5d0JuMd\nn4/w41oosXP3lYoj+kiViARHiU1EgqPEJiLBUWITkeAosYlIcJTYRCQ4SmwiEhwlNhEJTlUKdA9O\nNXrHxwr+rrQuxYs/Y7kpGpMv8Ycbi/mLAvs38o60d3yCd5OtT/JCyC0D/d7xe//oT+gcf/NExY/w\nnjc2zI/dqamSdzyTOUznSIFXZU4s8JjDg5U/IwgAyPIOr66bdzDu7OEFrUXS1CZmvOC41MDvp2Qp\nGpMnBehTRb6WhiS/n/okL9Cdy/o7WueTfC2ulKcxleiKTUSCo8QmIsFRYhOR4CixiUhwlNhEJDhK\nbCISHCU2EQlOlerY/Pnz7//lde/4TZu76X30pZppTGOKP9yNfX3+8W6+y/tV2/iu3XC81ur0mXHv\n+F8/ymvUdr+2n8Zk02zHbSDPysscb2jp6M7eQLGOH99izF8DlYC/bhIAChEacBZifJ56Vo5Fmj8C\nQDrLry9chOaOiYR/1/l4yV+LCAClDK8jLGT4PKmS/zEljD/mbJ4/5kp0xSYiwVFiE5HgKLGJSHCU\n2EQkOEpsIhIcJTYRCY4Sm4gER4lNRIJTlQLd2Zi/ed0zuw96xw+9dYTexz03f4jGXLWR77h99Mgh\n7/idt3yEzlEfoYnebI4XiD72y5e846/sP0XnmM/X0RiQwk4AiCf9vwOLJX+TQwCIGd/N3MV4UWax\n5G/SmSXFoQCQL/JGn2a8iDoL/7ntHD8uCXJsASAe5zFNjf7nOkqjzyKvvUUxQvPMApmokOdNJOta\neIPZSnTFJiLBUWITkeAosYlIcJTYRCQ4SmwiEhwlNhEJjhKbiASHJjYz6zez583sDTPbZ2Z/Xr79\nATMbMrPXyv/dffmXKyLCRSnQLQD4hnNur5k1A3jVzJ4rjz3onHuQTbCu2797+sTZEe/46bOTdJG/\nee0AjSnmN9MYkILL9X2b6AyxOC+K/e2u39GYJ597wTueLfEdxJGMUHwbW/mFezHDu+O6CEW8JVJ8\nC/Ci12KErrXJBD/1Lc6LqJHwny+JCHPE43wtra0tfB7yPMYcf56LEWJK4AW6KPgLdDf09dIpWtr8\nBfW+sn56RJ1zIwBGyl/PmdmbADaWhy+9d6+IyGXynn5Vm9kAgJ0AdpVv+pqZ7TWzH5kZb1YvIlIF\nkRNb+WXozwB83Tk3B+D7ALY453Zi8YqOviQVEamGSB+CN7MEFpPao865JwHAOTe2JOSHAH5R6d9P\nvPLObkoNG3agccPVl7RYEfngmji+D2eP74sUG7W7x48BHHDOfe/cDWbWW37/DQA+D6DiPm9dH703\n4t2IiCyva/P16Np8/fnvj/z6pxVjaWIzs9sA/DGAfWa2B4AD8B0A95nZTgAlAIMAvrqiVYuIrJIo\nfxV9EcByf7N+evWXIyKycvrkgYgEpyoddJOkSDFFikjzxgtej41M05jM/Bs05vc/5v/DRmPHBjrH\nVIa3If3nf91FYzLO3/E0l+cdXuvredfaUomvd35+nsYwcYtQFBulMpLU+dZFKHi1eIQi01iEeeoa\nveONDf5xAEgk+f3kc7z77UxmzjsepctxlhTWAkB7ZzeN6dvoj2mu58c/PTtDYyrRFZuIBEeJTUSC\no8QmIsFRYhOR4CixiUhwlNhEJDhKbCISnKrUsRULpHkgaW5XirBTeW7ZD0dcaHSWN0PcfWDIO/7Z\nBV4LNON4/c3QJI+pa/Y3Fyws8MeczmRoTFPTymutMll+bC3G1xszHpNM+GugXIQ6Nhfhd3qqjp93\ns3n/uZ0r+GvLAKAhQq1blB3lWQ3aXJrXPbZ08Bq19vW8ljNb8J8PJw/wmtJUhKajleiKTUSCo8Qm\nIsFRYhOR4CixiUhwlNhEJDhKbCISHCU2EQmOEpuIBKcqBbpgjQydfzwRoSlgyfHCzmKEeY6NznrH\nf/DYU3SOT9/1cRpzdOgMjVkoksJl8ALSVANv0hlP+XczB4CmuH8tdQXe0HJhhher5vO8oaIjhajJ\nBn5axxP8fMlFWEucNFEtRWjumF7wn3NR52Fr6ejsonOs6+PFt2PjEzRmcuy0d3xq8BCdY/vWrTSm\nEl2xiUhwlNhEJDhKbCISHCU2EQmOEpuIBEeJTUSCo8QmIsFRYhOR4FSlQLervcM7nkn7CxSjdP6s\nj/MupIU87yYbS/kLWv/vb/fSOY6dOkVjpubzNGZidsE7XuCHBc1N/i68ANAUYSf4+jr/cUlEKPJt\naORFvPEIXXYTSf99FSP8vi5EKHi1CDHO+bu8FvP8ec7l+BPZ0MCLsbvXrfOOd3bz4ttchEL3bIqn\njTQ5X0oJfr7MpdM0phJdsYlIcJTYRCQ4SmwiEhwlNhEJjhKbiARHiU1EgqPEJiLBoYnNzOrMbJeZ\n7TGzN8zsv5Zv7zCzZ83skJk9Y2Ztl3+5IiIcrbRzzmXN7E7n3IKZxQG8aGa3AfgsgF855/7KzL4J\n4NsAvrXcHNmMv9CujqTXbIEXOSbjvNtpIUI5sov5FxNr5AWvxyN0x41F6OBayPsLRAt5XlibzvCi\n5Ll53tk2RgpnWQEvADSleAfjxghFvLGY/3GnIhSzNjY205hcjp9TZyb83WRL4HMkUvyFU2crX29v\nl78QfkMf76A7OZ+lMTOTZ2nM7NSkd7y9i69l7MwYjakk0ktR59y5Evi68r+ZBHAvgEfKtz8C4HOX\nvAoRkVUUKbGZWczM9gAYAfBr59wBAD3OuVEAcM6NAFh/+ZYpIhJdpM+KOudKAG4ws1YAz5jZHQAu\nfp3EP1gnIlIF7+lD8M65GTP7JwAfBTBqZj3OuVEz6wVQ8Y2l07v+z/mvWzZeg5b+ay51vSLyAVWa\nGUZpdjhSLE1sZrYOQN45N21mDQA+BeAvADwF4MsAvgvgSwCerDTHho9/PtJiREQqibX2Idbad/77\n4uk9FWOjXLH1AXjEzAyL78k96pz75/J7bk+Y2Z8COAHgCytatYjIKolS7rEPwI3L3H4WwCcvx6JE\nRFaiKo0ms6RhXH3cvOMFXv6EUo43pbMIj7YEf41UqeRvLLg4R4QatVyEJoZF/3FxLkojRB5TitBo\nktWxnT3La5sm8vw5amtp4jEd/hqoNrJrPQC4Rl7rVijxmq6k+c+HRB0/FzJpXmtYl/SfC1HWkl+Y\npnMU5vlaZifHaUwp7z92DXW80WSG7Gzvo49UiUhwlNhEJDhKbCISHCU2EQlO1RPb/PChat/limTG\nBtd6CZEVZ0bWegnvSWnu0j/kvBZmTh9d6yVENjb45lov4T3JTZxY1fmU2Ijs2Ooe8MupNDO61kt4\nT9wc/+taLXk/JbbxE0psIiJBUWITkeBYlALOFd2Bmbp+iMhl4ZxbtnL5sic2EZFq00tREQmOEpuI\nBKeqic3M7jazg2Z2uLwBTE0zs0Eze728Q9fLa72epczsITMbNbPfLbmtZncOq7DeB8xsyMxeK/93\n91qu8Rwz6zez58u7su0zs/9cvr0mj+8y6/3z8u01d3yrtetd1d5jM7MYgMMA7gJwGsBuAF90zh2s\nygIugZkdA3CTc86/5c4aMLPbAcwB+Ilz7sPl274LYGLJzmEdzrlldw6rtgrrfQDArHPuwTVd3EXK\nHaF7nXN7zawZwKtY3LzoftTg8fWs9z+iNo9v49Jd7wD8Fyzuerdqx7aaV2w3A3jLOXfCOZcH8FMs\nHvxadq4bZAh6AAACKElEQVS5Zs1xzr2Axd3ClqrZncMqrBdYPMY1xTk34pzbW/56DsCbAPpRo8e3\nwno3lodr8fhe9l3vqvlDuxHAySXfD+Gdg1+rHIDnzGy3mX1lrRcTwfr34c5hXzOzvWb2o1p5abeU\nmQ0A2AngJbwPdmZbst5d5Ztq7vhWY9e7mrwaqSG3OeduBPAHAP6s/HLq/aTWa3m+D2CLc24nFk/y\nWnvJ1AzgZwC+Xr4Squmd2ZZZb00eX+dcyTl3Axavgj9xOXa9q2ZiOwXgiiXf95dvq1nOueHy/8cA\n/ByLL6dr2aiZ9QDn33fhW9KvIefcmHvnTd4fAvjYWq5nKTNLYDFJPOqcO7dRUc0e3+XWW8vHF1jc\n9Q7ABbveAatzbKuZ2HYD2GZmV5pZCsAXsbjTVU0ys8byb0CYWROATwPYv7arehfDhe+hnNs5DCA7\nh62RC9ZbPoHP+Txq6/j+GMAB59z3ltxWy8f3XeutxeNrZuvOvSResuvdHqzysa3qJw/Kf27+HhYT\n6kPOub+s2p2/R2a2GYtXaQ6Le0M8VkvrNbPHAdwBoAvAKIAHAPwDgL8DsAnlncOcc1NrtcalKqz3\nTiy+H1QCMAjgq+feZ1lLZnYbgP8HYB8Wn38H4DsAXgbwBGrs+HrWex9q7Pia2fVY/OPA0l3v/puZ\ndWIVj60+UiUiwdEfD0QkOEpsIhIcJTYRCY4Sm4gER4lNRIKjxCYiwVFiE5HgKLGJSHD+P0+f06AU\nGSm7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a2923b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = np.load('cifar10.npz')\n",
    "mean_image = np.load('mean_image.npy')\n",
    "data_zeromean = data['raw'] - mean_image\n",
    "# print(np.shape(data['whitened'][0:1]))\n",
    "# print(np.amax(data['raw'][0:1]))\n",
    "# print(np.amin(data['raw'][0:1]))\n",
    "##\n",
    "def make_image(X):\n",
    "    im = np.swapaxes(X.T, 0, 1)\n",
    "    im = im - im.min()\n",
    "    im = im * 1.0 / im.max()\n",
    "    return im\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(make_image(data['raw'][0]), interpolation='nearest')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (CIFAR10 Layer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "FPGA_net = {}\n",
    "FPGA_net['input'] = InputLayer((None, 3, 32, 32))\n",
    "FPGA_net['cifar10'] = FPGA_CIFAR10(FPGA_net['input'])\n",
    "FPGA_net['prob'] = NonlinearityLayer(FPGA_net['cifar10'], softmax)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  3.339431673000007\n",
      "CPU times: user 11.2 s, sys: 3.35 s, total: 14.6 s\n",
      "Wall time: 12.6 s\n"
     ]
    }
   ],
   "source": [
    "batch_size = 500\n",
    "\n",
    "%time prob = lasagne.layers.get_output(FPGA_net['cifar10'], floatX(data_zeromean[0:batch_size]), deterministic=True)#.eval()\n",
    "FPGA_predicted = np.argmax(prob, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3 8 0 0 6 6 9 6 7 1 0 9 5 2 9 8 5 9 8 6 0 6 0 9 4 6 4 0 9 6 6 5 0 3 9 1 4\n",
      " 1 9 3 0 6 3 6 0 9 3 0 7 0 9 9 2 6 8 8 4 2 0 3 7 3 3 9 1 2 1 0 3 7 0 6 0 8\n",
      " 0 2 0 5 2 1 8 1 1 0 0 7 0 0 8 1 0 1 9 6 0 6 6 0 6 2 0 1 6 3 1 1 1 6 6 2 4\n",
      " 0 6 0 1 7 0 4 1 5 8 0 9 2 8 1 9 3 3 8 4 1 8 9 0 0 9 7 6 1 0 0 1 1 0 0 6 3\n",
      " 1 3 0 9 0 0 2 5 0 9 3 4 0 1 6 6 1 0 0 6 7 8 9 0 1 8 9 1 3 7 5 0 0 5 2 0 0\n",
      " 8 6 3 7 0 0 0 0 2 0 0 8 0 5 8 0 1 1 9 1 0 0 2 7 9 7 0 3 5 0 8 0 9 0 4 3 5\n",
      " 1 1 3 1 0 4 0 0 3 1 0 8 1 0 0 0 0 1 1 1 1 9 0 0 1 8 7 4 7 3 1 1 2 0 3 0 8\n",
      " 9 8 9 5 6 1 0 8 1 7 0 0 0 6 9 8 0 0 0 0 6 9 0 0 1 9 9 1 0 3 9 1 2 6 1 3 0\n",
      " 6 1 0 6 6 2 0 8 3 1 9 0 1 0 1 6 0 0 4 0 0 5 6 5 0 5 2 0 2 1 7 0 4 6 0 7 0\n",
      " 6 6 1 3 8 0 1 4 6 0 7 8 3 4 6 8 9 1 1 0 2 0 7 0 2 1 8 9 5 8 8 9 9 0 7 0 1\n",
      " 0 7 9 0 7 7 9 7 0 6 6 9 9 0 5 5 7 4 3 5 1 0 6 2 1 6 2 0 0 5 9 1 7 8 8 1 0\n",
      " 0 6 2 7 0 3 6 1 9 3 2 0 7 4 7 8 0 0 9 1 1 6 6 0 0 6 7 0 7 9 6 3 1 1 0 9 9\n",
      " 6 6 0 2 0 0 9 9 7 0 9 4 6 2 0 9 4 6 1 1 9 4 1 3 3 8 0 0 8 0 6 5 6 0 5 4 9\n",
      " 6 8 0 3 9 9 9 0 0 1 0 0 1 1 1 0 2 2 0]\n",
      "[3 8 8 0 6 6 1 6 3 1 0 9 5 7 9 8 5 7 8 6 7 0 4 9 5 2 4 0 9 6 6 5 4 5 9 2 4\n",
      " 1 9 5 4 6 5 6 0 9 3 9 7 6 9 8 0 3 8 8 7 7 4 6 7 3 6 3 6 2 1 2 3 7 2 6 8 8\n",
      " 0 2 9 3 3 8 8 1 1 7 2 5 2 7 8 9 0 3 8 6 4 6 6 0 0 7 4 5 6 3 1 1 3 6 8 7 4\n",
      " 0 6 2 1 3 0 4 2 7 8 3 1 2 8 0 8 3 5 2 4 1 8 9 1 2 9 7 2 9 6 5 6 3 8 7 6 2\n",
      " 5 2 8 9 6 0 0 5 2 9 5 4 2 1 6 6 8 4 8 4 5 0 9 9 9 8 9 9 3 7 5 0 0 5 2 2 3\n",
      " 8 6 3 4 0 5 8 0 1 7 2 8 8 7 8 5 1 8 7 1 3 0 5 7 9 7 4 5 9 8 0 7 9 8 2 7 6\n",
      " 9 4 3 9 6 4 7 6 5 1 5 8 8 0 4 0 5 5 1 1 8 9 0 3 1 9 2 2 5 3 9 9 4 0 3 0 0\n",
      " 9 8 1 5 7 0 8 2 4 7 0 2 3 6 3 8 5 0 3 4 3 9 0 6 1 0 9 1 0 7 9 1 2 6 9 3 4\n",
      " 6 0 0 6 6 6 3 2 6 1 8 2 1 6 8 6 8 0 4 0 7 7 5 5 3 5 2 3 4 1 7 5 4 6 1 9 3\n",
      " 6 6 9 3 8 0 7 2 6 2 5 8 5 4 6 8 9 9 1 0 2 2 7 3 2 8 0 9 5 8 1 9 4 1 3 8 1\n",
      " 4 7 9 4 2 7 0 7 0 6 6 9 0 9 2 8 7 2 2 5 1 2 6 2 9 6 2 3 0 3 9 8 7 8 8 4 0\n",
      " 1 8 2 7 9 3 6 1 9 0 7 3 7 4 5 0 0 2 9 3 4 0 6 2 5 3 7 3 7 2 5 3 1 1 4 9 9\n",
      " 5 7 5 0 2 2 2 9 7 3 9 4 3 5 4 6 5 6 1 4 3 4 4 3 7 8 3 7 8 0 5 7 6 0 5 4 8\n",
      " 6 8 5 5 9 9 9 5 0 1 0 8 1 1 8 0 2 2 0]\n",
      "0.496\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == data['labels'][0:batch_size])\n",
    "print(FPGA_predicted)\n",
    "print(data['labels'][0:batch_size])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (QuickTest Function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  0.00503736900009244\n"
     ]
    },
    {
     "ename": "TimeoutError",
     "evalue": "DMA wait timed out.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTimeoutError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-30-ec1d3961b0a6>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mOFMDim\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mOFMCH\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmagic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'time FPGA_output = FPGAQuickTest(data_zeromean*0.85, batch_size, OFMDim, OFMCH)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0mFPGA_predicted\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mFPGA_output\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;31m#np.save('FPGA_output', FPGA_output)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/interactiveshell.py\u001b[0m in \u001b[0;36mmagic\u001b[0;34m(self, arg_s)\u001b[0m\n\u001b[1;32m   2144\u001b[0m         \u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marg_s\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m' '\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2145\u001b[0m         \u001b[0mmagic_name\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmagic_name\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlstrip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprefilter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mESC_MAGIC\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2146\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2147\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2148\u001b[0m     \u001b[0;31m#-------------------------------------------------------------------------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/interactiveshell.py\u001b[0m in \u001b[0;36mrun_line_magic\u001b[0;34m(self, magic_name, line)\u001b[0m\n\u001b[1;32m   2065\u001b[0m                 \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'local_ns'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msys\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getframe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstack_depth\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf_locals\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2066\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuiltin_trap\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2067\u001b[0;31m                 \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2068\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2069\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<decorator-gen-59>\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/magic.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(f, *a, **k)\u001b[0m\n\u001b[1;32m    186\u001b[0m     \u001b[0;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    187\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 188\u001b[0;31m         \u001b[0mcall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    189\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    190\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/magics/execution.py\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m   1174\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1175\u001b[0m             \u001b[0mst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1176\u001b[0;31m             \u001b[0mexec\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mglob\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlocal_ns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1177\u001b[0m             \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1178\u001b[0m             \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/CIFAR_10/conv_fpga.py\u001b[0m in \u001b[0;36mFPGAQuickTest\u001b[0;34m(test_data, batch_size, OFMDim, OFMCH)\u001b[0m\n\u001b[1;32m    294\u001b[0m     \u001b[0minput_val\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m28\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput_val\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    295\u001b[0m     \u001b[0minput_val\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minput_val\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mastype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 296\u001b[0;31m     \u001b[0moutput_fpga\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdouble\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_val\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhw_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    297\u001b[0m     \u001b[0;31m#output_mat_tmp = output_fpga[8:].reshape(batch_size, OFMDim*OFMDim, OFMCH)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    298\u001b[0m     \u001b[0;31m#output_mat_tmp = output_mat_tmp.transpose(0, 2, 1)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/CIFAR_10/conv_fpga.py\u001b[0m in \u001b[0;36mhw_value\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     39\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0mhw_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 40\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mexecute_hardware\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     42\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m__str__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/CIFAR_10/conv_fpga.py\u001b[0m in \u001b[0;36mexecute_hardware\u001b[0;34m(plan)\u001b[0m\n\u001b[1;32m    181\u001b[0m   \u001b[0mend_time\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprocess_time\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    182\u001b[0m   \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Elapsed Test Time: \"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_time\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mstart_time\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 183\u001b[0;31m   \u001b[0mret_dma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    184\u001b[0m   \u001b[0mbytes_read\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mret_dma_mmio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0x58\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    185\u001b[0m   \u001b[0mffi\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpynq\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrivers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mffi\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/pynq/drivers/dma.py\u001b[0m in \u001b[0;36mwait\u001b[0;34m(self, wait_timeout)\u001b[0m\n\u001b[1;32m    439\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mseconds\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwait_timeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror_message\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    440\u001b[0m             \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 441\u001b[0;31m                 \u001b[0;32mif\u001b[0m \u001b[0mlibdma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mXAxiDma_Busy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDMAengine\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdirection\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    442\u001b[0m                     \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    443\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/pynq/drivers/dma.py\u001b[0m in \u001b[0;36mhandle_timeout\u001b[0;34m(self, signum, frame)\u001b[0m\n\u001b[1;32m    173\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    174\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mhandle_timeout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msignum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mframe\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 175\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mTimeoutError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0merror_message\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    176\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    177\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__enter__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTimeoutError\u001b[0m: DMA wait timed out."
     ]
    }
   ],
   "source": [
    "batch_size = 2\n",
    "\n",
    "OFMDim = 1\n",
    "OFMCH = 10\n",
    "%time FPGA_output = FPGAQuickTest(data_zeromean*0.85, batch_size, OFMDim, OFMCH)\n",
    "FPGA_predicted = np.argmax(FPGA_output.reshape(batch_size, -1), 1)\n",
    "#np.save('FPGA_output', FPGA_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.53\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == data['labels'][0:batch_size])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testbench: Check output for each layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "upper bound 9.44879023774\n",
      "lower bound -6.40481699552\n",
      "(1, 10)\n",
      "[[-0.35958596 -6.404817    0.11735196  9.44879024  1.85489804  7.99725445\n",
      "   1.80384364  2.411506    1.85896584 -3.1019715 ]]\n"
     ]
    }
   ],
   "source": [
    "conv_output = lasagne.layers.get_output(net['ip2'], floatX(data['raw'][0:1]), deterministic=True).eval()\n",
    "print('upper bound', np.amax(conv_output))\n",
    "print('lower bound', np.amin(conv_output))\n",
    "print(conv_output.shape)\n",
    "print(conv_output)\n",
    "# np.allclose(FPGA_output, conv_output)\n",
    "# np.save('cpu_output', conv_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A [[[-1.44444444]]\n",
      "\n",
      " [[-4.11111111]]\n",
      "\n",
      " [[-1.05555556]]\n",
      "\n",
      " [[-1.5       ]]\n",
      "\n",
      " [[ 9.5       ]]\n",
      "\n",
      " [[-3.33333333]]\n",
      "\n",
      " [[-1.83333333]]\n",
      "\n",
      " [[-1.        ]]\n",
      "\n",
      " [[ 0.94444444]]\n",
      "\n",
      " [[ 5.16666667]]]\n",
      "B [ -2.20250484  -5.73974968  -2.03936998  -2.61254005  13.50087001\n",
      "  -4.67510671  -2.59636619  -0.8826095    1.86436696   7.05478231]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.load('FPGA_output.npy')\n",
    "B = np.load('cpu_output.npy')\n",
    "print('A',A[4])\n",
    "print('B',B[4])\n",
    "np.allclose(A,B)\n",
    "#print(np.amax(B))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight shape (20, 1, 5, 5)\n",
      "input shape (1, 1, 28, 28)\n",
      "[[[[ -1.23521651e+00  -2.04537287e+00  -6.32075214e-01 ...,\n",
      "     -1.63300213e+00  -2.55450387e+00  -2.07054086e+00]\n",
      "   [ -1.40137811e+00  -1.77353978e+00  -1.84130565e+00 ...,\n",
      "     -1.33434864e+00  -2.22999067e+00  -2.42328846e+00]\n",
      "   [ -1.26774366e+00  -1.71084404e+00  -1.31209231e+00 ...,\n",
      "     -2.24487298e+00  -2.73626462e+00  -1.29489929e+00]\n",
      "   ..., \n",
      "   [ -1.62035386e+00  -7.08239809e-01  -2.27327630e+00 ...,\n",
      "     -1.44700859e+00  -3.67266482e+00  -3.81733384e+00]\n",
      "   [ -2.17358027e+00  -1.13402332e+00  -3.92598738e-03 ...,\n",
      "     -2.06156307e+00  -1.44563193e+00  -2.10430690e+00]\n",
      "   [ -1.27752032e+00  -1.06594470e+00  -1.12218224e+00 ...,\n",
      "     -1.30819874e-01  -1.01351195e+00  -1.54660986e+00]]\n",
      "\n",
      "  [[ -2.60145689e-01  -5.13031884e-02   1.01291686e+00 ...,\n",
      "      8.97090336e-01   6.42144868e-01   1.94622497e-01]\n",
      "   [ -1.03464390e+00  -1.08955611e-01   4.27987106e-02 ...,\n",
      "      2.66289295e-01  -8.85309893e-02   4.17684471e-01]\n",
      "   [ -5.75026525e-01   9.64486380e-01  -2.50498932e-01 ...,\n",
      "     -1.69981671e-01   3.83626229e-02  -4.29510827e-01]\n",
      "   ..., \n",
      "   [ -5.05453794e-01  -8.63296574e-01  -9.17132820e-02 ...,\n",
      "      3.77458619e-01  -1.95137555e-01  -5.17759730e-01]\n",
      "   [ -2.58788723e-01  -9.64247429e-01  -7.14861086e-02 ...,\n",
      "     -1.42263971e-01   1.91776053e-01   5.46139432e-01]\n",
      "   [ -2.64195517e-01   6.57731518e-01   9.02944947e-02 ...,\n",
      "     -3.68829995e-01   3.83041570e-01  -2.01499939e-01]]\n",
      "\n",
      "  [[ -3.65994417e+00  -5.14173995e+00  -4.96577030e+00 ...,\n",
      "     -4.51097770e+00  -4.49018481e+00  -4.99095697e+00]\n",
      "   [ -4.11705978e+00  -3.84009698e+00  -4.76372528e+00 ...,\n",
      "     -5.18592852e+00  -5.22249345e+00  -4.80849165e+00]\n",
      "   [ -3.97032701e+00  -3.38993575e+00  -4.60564690e+00 ...,\n",
      "     -3.40043616e+00  -4.73042337e+00  -5.62832834e+00]\n",
      "   ..., \n",
      "   [ -4.71954725e+00  -2.77478101e+00  -1.26896644e+00 ...,\n",
      "     -3.80103310e+00  -3.90681893e+00  -3.88966040e+00]\n",
      "   [ -4.00748279e+00  -3.81785394e+00  -4.35615060e+00 ...,\n",
      "     -4.77401208e+00  -5.17462886e+00  -4.66860414e+00]\n",
      "   [ -3.03090842e+00  -1.37022436e+00  -2.61072677e+00 ...,\n",
      "     -3.64930527e+00  -3.39860399e+00  -3.79809401e+00]]\n",
      "\n",
      "  ..., \n",
      "  [[ -1.68899562e-01  -4.70391744e-01  -6.97371397e-01 ...,\n",
      "     -5.28510538e-01  -8.69725970e-01  -9.40058672e-01]\n",
      "   [ -1.13698729e-01  -2.39520724e-01  -1.05513776e+00 ...,\n",
      "      1.89672299e-02  -5.85293545e-01  -1.24348531e+00]\n",
      "   [  4.84735384e-01  -2.85855296e-01  -2.31752558e-01 ...,\n",
      "      2.45801093e-01  -1.48229439e+00  -3.27884453e-01]\n",
      "   ..., \n",
      "   [ -1.15616940e+00  -3.67509939e-01  -1.53234489e+00 ...,\n",
      "      3.00053739e-01  -2.14880543e+00  -1.62754632e-01]\n",
      "   [  3.90369861e-01  -1.21054544e+00  -2.43554270e-02 ...,\n",
      "     -1.52735065e+00   1.01189873e+00  -9.24407381e-01]\n",
      "   [ -6.92695001e-01   7.25373323e-01  -4.75248336e-01 ...,\n",
      "      8.00767456e-01   1.86470517e-01  -1.68280526e+00]]\n",
      "\n",
      "  [[  1.00908039e+00   1.18807713e+00   1.30922008e+00 ...,\n",
      "      1.39107018e+00   2.26549537e+00   2.93546801e+00]\n",
      "   [  3.13643280e+00   1.43321484e+00   1.99360973e+00 ...,\n",
      "      1.85552427e+00   9.63954788e-01   1.18764058e+00]\n",
      "   [  1.73749805e+00   2.13676041e+00   2.66441862e+00 ...,\n",
      "      2.95361881e+00   2.92212581e+00   2.39243630e+00]\n",
      "   ..., \n",
      "   [  2.37215457e+00   5.14820833e-01   1.65617652e-01 ...,\n",
      "      1.73166437e+00   1.92869916e+00   2.61471371e+00]\n",
      "   [  2.93004702e+00   1.20636201e+00   1.77732324e+00 ...,\n",
      "      1.48198130e+00   1.97564921e+00   1.10872634e+00]\n",
      "   [  8.97655375e-01   1.19172669e+00   1.02551056e+00 ...,\n",
      "      1.31002310e+00   1.73514415e+00   4.43609674e-01]]\n",
      "\n",
      "  [[  1.56064707e-01   4.11192449e-02  -1.37242171e+00 ...,\n",
      "      2.52051019e-01  -3.18872694e-01  -7.55935752e-01]\n",
      "   [  1.35216298e-01   6.02522093e-01   7.14582454e-01 ...,\n",
      "      2.80997509e-01  -2.69425698e-01   1.67536314e+00]\n",
      "   [  9.27890678e-01   1.72920391e+00   5.26195342e-01 ...,\n",
      "      1.34184756e+00   1.13691001e+00  -9.60607615e-01]\n",
      "   ..., \n",
      "   [  2.85855771e-01  -5.08133719e-01   1.19018943e-01 ...,\n",
      "      1.65176111e-01   7.94100231e-01   5.13618884e-01]\n",
      "   [  9.61006454e-01   1.08911135e+00  -6.81093900e-02 ...,\n",
      "     -8.57343171e-02  -9.72893105e-02  -1.80669722e-01]\n",
      "   [  8.79052882e-01  -4.41791561e-01   1.12758501e+00 ...,\n",
      "      1.81297062e+00  -1.18995210e+00  -6.84469087e-02]]]]\n",
      "max 4.78987174592\n",
      "min -6.67566047666\n",
      "##############################\n",
      "weight shape (20, 1, 5, 5)\n",
      "kermax 0.996485495516\n",
      "kermin -0.998557337864\n",
      "[[[[-1.16666667 -1.96666667 -0.5        ..., -1.56666667 -2.43333333\n",
      "    -1.96666667]\n",
      "   [-1.3        -1.63333333 -1.7        ..., -1.2        -2.13333333\n",
      "    -2.33333333]\n",
      "   [-1.13333333 -1.63333333 -1.23333333 ..., -2.16666667 -2.63333333 -1.2       ]\n",
      "   ..., \n",
      "   [-1.56666667 -0.6        -2.13333333 ..., -1.36666667 -3.56666667\n",
      "    -3.66666667]\n",
      "   [-2.06666667 -1.03333333  0.         ..., -2.         -1.3        -2.03333333]\n",
      "   [-1.23333333 -1.03333333 -1.03333333 ..., -0.06666667 -0.96666667\n",
      "    -1.43333333]]\n",
      "\n",
      "  [[-0.23333333 -0.06666667  0.96666667 ...,  0.86666667  0.63333333\n",
      "     0.16666667]\n",
      "   [-1.03333333 -0.1         0.03333333 ...,  0.23333333 -0.03333333  0.4       ]\n",
      "   [-0.53333333  0.93333333 -0.23333333 ..., -0.13333333  0.03333333\n",
      "    -0.43333333]\n",
      "   ..., \n",
      "   [-0.5        -0.83333333 -0.06666667 ...,  0.33333333 -0.16666667 -0.5       ]\n",
      "   [-0.23333333 -0.9        -0.06666667 ..., -0.13333333  0.16666667\n",
      "     0.53333333]\n",
      "   [-0.26666667  0.6         0.06666667 ..., -0.33333333  0.36666667 -0.2       ]]\n",
      "\n",
      "  [[-3.46666667 -4.93333333 -4.73333333 ..., -4.33333333 -4.26666667\n",
      "    -4.76666667]\n",
      "   [-3.96666667 -3.66666667 -4.56666667 ..., -5.         -5.03333333\n",
      "    -4.63333333]\n",
      "   [-3.73333333 -3.13333333 -4.36666667 ..., -3.16666667 -4.5        -5.43333333]\n",
      "   ..., \n",
      "   [-4.53333333 -2.6        -1.13333333 ..., -3.6        -3.7        -3.66666667]\n",
      "   [-3.83333333 -3.63333333 -4.13333333 ..., -4.56666667 -4.96666667\n",
      "    -4.46666667]\n",
      "   [-2.8        -1.23333333 -2.5        ..., -3.46666667 -3.2        -3.6       ]]\n",
      "\n",
      "  ..., \n",
      "  [[-0.13333333 -0.46666667 -0.66666667 ..., -0.53333333 -0.83333333 -0.9       ]\n",
      "   [-0.06666667 -0.2        -1.         ...,  0.03333333 -0.56666667\n",
      "    -1.23333333]\n",
      "   [ 0.46666667 -0.26666667 -0.2        ...,  0.2        -1.43333333 -0.3       ]\n",
      "   ..., \n",
      "   [-1.2        -0.33333333 -1.46666667 ...,  0.3        -2.06666667 -0.1       ]\n",
      "   [ 0.36666667 -1.16666667 -0.03333333 ..., -1.56666667  1.03333333 -0.9       ]\n",
      "   [-0.66666667  0.7        -0.36666667 ...,  0.76666667  0.2        -1.66666667]]\n",
      "\n",
      "  [[ 0.9         1.1         1.2        ...,  1.33333333  2.16666667\n",
      "     2.83333333]\n",
      "   [ 3.03333333  1.33333333  1.9        ...,  1.73333333  0.9         1.13333333]\n",
      "   [ 1.63333333  2.06666667  2.6        ...,  2.86666667  2.83333333\n",
      "     2.33333333]\n",
      "   ..., \n",
      "   [ 2.3         0.5         0.1        ...,  1.6         1.86666667\n",
      "     2.53333333]\n",
      "   [ 2.83333333  1.13333333  1.7        ...,  1.36666667  1.9         1.03333333]\n",
      "   [ 0.8         1.13333333  0.96666667 ...,  1.2         1.66666667\n",
      "     0.36666667]]\n",
      "\n",
      "  [[ 0.16666667  0.03333333 -1.4        ...,  0.23333333 -0.33333333\n",
      "    -0.73333333]\n",
      "   [ 0.1         0.6         0.7        ...,  0.26666667 -0.23333333\n",
      "     1.66666667]\n",
      "   [ 0.96666667  1.7         0.53333333 ...,  1.33333333  1.1        -0.93333333]\n",
      "   ..., \n",
      "   [ 0.26666667 -0.5         0.1        ...,  0.1         0.83333333  0.5       ]\n",
      "   [ 0.86666667  1.06666667  0.         ..., -0.1        -0.1        -0.13333333]\n",
      "   [ 0.86666667 -0.43333333  1.06666667 ...,  1.8        -1.13333333\n",
      "    -0.06666667]]]]\n",
      "max 4.73333333333\n",
      "min -6.53333333333\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Testbench for a single conv layer\n",
    "net1 = {}\n",
    "net1['input_1'] = InputLayer((None, 1, 28, 28))\n",
    "net1['conv_1'] = ConvLayer(net1['input_1'], num_filters=20, filter_size=5, pad=0, nonlinearity=linear)\n",
    "weight = np.random.rand(20, 1, 5, 5)*2-1\n",
    "#weight = weight[...,::-1,::-1]\n",
    "print('weight shape',weight.shape)\n",
    "\n",
    "net1['conv_1'].W.set_value(weight)\n",
    "input = np.random.rand(1, 1, 28, 28)\n",
    "print('input shape',input.shape)\n",
    "conv_output_test = lasagne.layers.get_output(net1['conv_1'], floatX(input), deterministic=True).eval()\n",
    "print(conv_output_test)\n",
    "print('max', np.amax(conv_output_test))\n",
    "print('min', np.amin(conv_output_test))\n",
    "print('##############################')\n",
    "FPGALoadW(weight, 1, 28, 24, 0)\n",
    "FPGA_output = FPGAQuickTest(input, 1, 24, 20)\n",
    "print(FPGA_output)\n",
    "print('max', np.amax(FPGA_output))\n",
    "print('min', np.amin(FPGA_output))\n",
    "np.allclose(conv_output_test, FPGA_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  4.87205496   0.           0.           0.          10.32939182   0.\n",
      "    3.73065467   0.           3.32166961   8.04596126   2.47765243\n",
      "    2.38071792   0.           0.          10.23824185  11.37844921   0.\n",
      "    2.49904565   6.67317322   7.72101975   0.          16.22842281\n",
      "    4.21925851   1.71412087  13.86398149   0.           6.47131831   0.\n",
      "    0.           0.          11.01161799  16.50425193  10.09142138   0.\n",
      "    8.71200305   0.          17.73560793   7.26002949   0.           0.\n",
      "    0.           0.           0.           3.64882534   8.84798804\n",
      "    0.51790838   0.          12.24283568   0.          10.07998283   0.\n",
      "   13.93915632   0.           1.31342381   2.37565862   0.18018827\n",
      "   15.77413368   0.           0.           3.52941475   4.3752688\n",
      "    0.96245774   5.99787461   0.           0.           0.           7.57239814\n",
      "   11.34722104   0.           7.44900567   8.85447856   0.           3.63706984\n",
      "    4.4351584    6.1741987    5.12098359   0.           0.           0.54486455\n",
      "    0.           2.34716758  11.65961668  14.37530636   2.73037676   0.\n",
      "   11.02818577   0.           1.05222788   0.           0.           0.\n",
      "   11.79985969   0.           2.18722952   8.07016441   0.           0.48128377\n",
      "    0.42685334   0.           0.           0.           7.97839663   0.\n",
      "    0.           0.58764638   8.87131068   0.          15.18896369   0.\n",
      "    0.           5.69908405   0.           0.           0.           0.\n",
      "    0.           8.1806746   10.6865666    5.4502488    0.          15.43516532\n",
      "   11.31803067   0.          15.8883904    0.           0.           0.\n",
      "    0.           0.           0.           0.           3.36289894\n",
      "   10.05100219   0.           0.           0.           0.           1.36648831\n",
      "    0.           2.34897277   0.           0.           0.16635098\n",
      "    2.03511467   0.           0.           0.82106747   0.          13.56543568\n",
      "    5.50418203   0.           6.48636786  14.66925314   0.           0.\n",
      "    9.72713839   4.6732693    0.66350285   6.13598948   0.           0.\n",
      "    0.           0.           7.71326117  21.66236202   6.07559027\n",
      "   11.30994027   0.03813952   0.           4.55986687   0.36733798\n",
      "   18.66398918   0.           5.73153027   0.           6.62628006   0.\n",
      "    0.           4.79217688   3.05292392   5.97852612   2.73634945   0.\n",
      "    0.           0.          11.81853997   0.           6.1895648    0.\n",
      "    0.           0.           0.          10.3059492    5.61476791\n",
      "    4.7638238    8.01105809   1.1456242    5.62021886   0.           0.38868603\n",
      "    1.33126799   0.42378301   0.           0.          12.38176997\n",
      "   10.35476552   0.           0.           0.          14.59775052   0.\n",
      "    0.           0.           5.85772319   0.           0.           0.\n",
      "    0.           0.78042736   0.           2.98850167   4.02550251\n",
      "    8.51495478   0.           0.           0.           0.           0.\n",
      "   13.9854325   11.85832479   0.           0.           9.23101527   0.\n",
      "   11.75796767   5.35753142   1.41690737  11.95718263  17.79470972\n",
      "    9.28500759   0.           0.           0.21147603   2.15857456   0.\n",
      "    0.          19.0844405   22.15722801   0.           4.17137518   0.\n",
      "    6.92154393   1.69509028  18.6839166    5.69489588   0.           0.\n",
      "    0.          11.88879818   5.15587561   0.          12.85335112\n",
      "    3.70553621   0.           0.           5.64413022   6.60557047\n",
      "    3.96497751   0.           4.60287163   0.           0.58812798   0.\n",
      "   11.1157155    0.           0.           0.           9.8155275    0.\n",
      "    0.          16.20489183   0.           0.           4.14233276   0.\n",
      "    4.34341638   8.46896904   0.           8.37185049  10.31071701   0.\n",
      "    0.10688457   0.           5.97306068   0.           4.89753857   0.\n",
      "    9.26196222   0.           9.58950062  13.38471899   2.16669255   0.\n",
      "    5.34838811   6.96073294   0.           0.           0.          15.08913938\n",
      "    2.51803672   7.88308158  12.21421887  10.20202114  10.73308962\n",
      "    3.52093679   0.           4.47425126   7.85916816   0.           0.\n",
      "    6.90158195   8.29529849   0.           0.           0.           0.\n",
      "   17.66852546   0.           0.           2.93399414   0.           0.\n",
      "    5.09620742   0.           0.           7.2849421    0.           0.\n",
      "    0.          10.18709824   2.5744383    0.           4.08406648   0.\n",
      "    6.83491966   9.35418875   0.           0.           0.           0.\n",
      "    4.95455747   0.           5.09100722   0.           8.41360688   0.\n",
      "    8.46274502   0.           6.89214509   6.67986715   0.           0.\n",
      "    0.           0.           2.28805299   0.           8.28019421\n",
      "   12.75343376   0.           0.           2.51653061   0.           0.82721538\n",
      "    0.           8.91417208   0.           5.68516714   3.11026239\n",
      "    7.29585777   0.           2.14074919   9.47591846   0.           0.\n",
      "   14.82286938   0.           0.           0.           0.           7.88781567\n",
      "   10.51508377   0.           0.           0.45894944   3.59168604   0.\n",
      "   14.03073171   6.76643939   0.           3.1323703   12.24137663\n",
      "    2.94689514   0.           0.           0.           7.25376755\n",
      "    3.42208125  19.07386738   0.           9.51545039   0.           0.\n",
      "   13.94424881  10.40733373   0.           0.           0.           0.\n",
      "    3.17219076   0.           5.35096481  10.96430379  11.77482422\n",
      "   11.30419229   0.           0.           0.          19.08975336\n",
      "    6.08621084   4.37618314   0.           1.98863682   7.90072457\n",
      "   11.825676     0.           0.28749063   0.           0.           0.\n",
      "    0.           0.           0.           6.91871997  15.02426763\n",
      "    3.41454367   6.40912401  11.50653594   0.           0.           0.31042584\n",
      "    0.           0.           3.18485489   0.           0.           0.\n",
      "    0.           0.           1.73935809   0.           4.51500803\n",
      "    9.51558738   4.15108236   4.26826456   0.           1.67253442\n",
      "    5.32168599   0.           4.73830834   0.           1.47740501   0.\n",
      "    0.           0.           0.           0.           0.          12.76195807\n",
      "    0.           0.           0.03329534   0.           0.48526704   0.\n",
      "    0.           3.19732557  20.76907562   0.           0.           1.19507225\n",
      "    0.           0.           3.39872442   0.           3.65393208\n",
      "   14.42782519  12.05240166   0.           0.           9.11073052]]\n",
      "max 22.157228007\n",
      "min 0.0\n",
      "##############################\n",
      "weight shape (500, 50, 4, 4)\n",
      "kermax 0.999999553699\n",
      "kermin -0.999996939193\n",
      "[[  4.8203125   0.          0.          0.         10.1328125   0.\n",
      "    3.7265625   0.          3.296875    8.0234375   2.421875    2.4765625\n",
      "    0.          0.         10.046875   11.3359375   0.          2.4765625\n",
      "    6.5234375   7.5625      0.         16.          4.140625    1.6484375\n",
      "   13.6953125   0.          6.4765625   0.          0.          0.\n",
      "   10.7734375  16.1171875  10.0703125   0.          8.59375     0.         17.5\n",
      "    7.28125     0.          0.          0.          0.          0.          3.6875\n",
      "    8.8828125   0.4140625   0.         12.2109375   0.          9.9296875\n",
      "    0.         13.6640625   0.          1.4921875   2.28125     0.1875\n",
      "   15.6484375   0.          0.          3.53125     4.234375    1.0234375\n",
      "    5.921875    0.          0.          0.          7.5625     11.2578125\n",
      "    0.          7.5078125   8.671875    0.          3.6328125   4.3359375\n",
      "    5.953125    5.0625      0.          0.          0.484375    0.\n",
      "    2.328125   11.4921875  14.296875    2.671875    0.         10.890625\n",
      "    0.          1.03125     0.          0.          0.         11.578125\n",
      "    0.          2.1015625   8.09375     0.          0.40625     0.4609375\n",
      "    0.          0.          0.          8.0078125   0.          0.\n",
      "    0.5390625   8.7421875   0.         15.03125     0.          0.\n",
      "    5.5703125   0.          0.          0.          0.          0.\n",
      "    8.03125    10.6875      5.3359375   0.         15.203125   11.2421875\n",
      "    0.         15.65625     0.          0.          0.          0.          0.\n",
      "    0.          0.          3.4453125   9.9765625   0.          0.          0.\n",
      "    0.          1.25        0.          2.4296875   0.          0.\n",
      "    0.2109375   1.9921875   0.          0.          0.7265625   0.\n",
      "   13.4453125   5.390625    0.          6.40625    14.4609375   0.          0.\n",
      "    9.546875    4.609375    0.703125    6.1484375   0.          0.          0.\n",
      "    0.          7.6640625  21.4765625   5.9296875  11.171875    0.078125\n",
      "    0.          4.453125    0.2734375  18.5         0.          5.6171875\n",
      "    0.          6.453125    0.          0.          4.7578125   2.9921875\n",
      "    5.890625    2.59375     0.          0.          0.         11.671875\n",
      "    0.          6.25        0.          0.          0.          0.\n",
      "   10.1171875   5.453125    4.75        7.8046875   1.2578125   5.6015625\n",
      "    0.          0.453125    1.234375    0.375       0.          0.\n",
      "   12.2890625  10.2109375   0.          0.          0.         14.4453125\n",
      "    0.          0.          0.          5.84375     0.          0.          0.\n",
      "    0.          0.7578125   0.          2.9609375   3.984375    8.3515625\n",
      "    0.          0.          0.          0.          0.         13.7890625\n",
      "   11.828125    0.          0.          9.21875     0.         11.6328125\n",
      "    5.3125      1.4140625  11.7890625  17.625       9.203125    0.          0.\n",
      "    0.1171875   2.1953125   0.          0.         18.9140625  21.875       0.\n",
      "    4.09375     0.          6.84375     1.6640625  18.4453125   5.7421875\n",
      "    0.          0.          0.         11.6953125   5.0703125   0.\n",
      "   12.5703125   3.6953125   0.          0.          5.6328125   6.53125\n",
      "    3.9453125   0.          4.59375     0.          0.5625      0.\n",
      "   10.9453125   0.          0.          0.          9.546875    0.          0.\n",
      "   16.0859375   0.          0.          3.9921875   0.          4.2578125\n",
      "    8.3984375   0.          8.2890625  10.203125    0.          0.1484375\n",
      "    0.          5.8828125   0.          4.7734375   0.          9.171875\n",
      "    0.          9.546875   13.25        2.0703125   0.          5.1640625\n",
      "    6.7578125   0.          0.          0.         14.984375    2.5234375\n",
      "    7.71875    12.1328125  10.15625    10.640625    3.5         0.\n",
      "    4.4296875   7.640625    0.          0.          6.8046875   8.1953125\n",
      "    0.          0.          0.          0.         17.3671875   0.          0.\n",
      "    2.953125    0.          0.          5.0859375   0.          0.          7.3125\n",
      "    0.          0.          0.         10.0703125   2.5078125   0.\n",
      "    4.015625    0.          6.765625    9.2109375   0.          0.          0.\n",
      "    0.          4.859375    0.          4.9765625   0.          8.265625\n",
      "    0.          8.4453125   0.          6.828125    6.6328125   0.          0.\n",
      "    0.          0.          2.21875     0.          8.296875   12.65625\n",
      "    0.          0.          2.46875     0.          0.8203125   0.\n",
      "    8.9140625   0.          5.65625     3.203125    7.2265625   0.\n",
      "    2.171875    9.3203125   0.          0.         14.578125    0.          0.\n",
      "    0.          0.          7.8359375  10.375       0.          0.\n",
      "    0.4296875   3.6953125   0.         13.8984375   6.703125    0.\n",
      "    3.0546875  12.1171875   3.0078125   0.          0.          0.\n",
      "    7.03125     3.421875   18.9140625   0.          9.453125    0.          0.\n",
      "   13.8359375  10.3203125   0.          0.          0.          0.\n",
      "    3.1640625   0.          5.3671875  10.96875    11.6328125  11.234375\n",
      "    0.          0.          0.         18.8203125   6.0390625   4.375       0.\n",
      "    1.8359375   7.8515625  11.7109375   0.          0.265625    0.          0.\n",
      "    0.          0.          0.          0.          6.875      14.78125\n",
      "    3.4921875   6.3984375  11.3203125   0.          0.          0.3125      0.\n",
      "    0.          3.140625    0.          0.          0.          0.          0.\n",
      "    1.7421875   0.          4.40625     9.4140625   4.0859375   4.15625\n",
      "    0.          1.6328125   5.3359375   0.          4.5703125   0.\n",
      "    1.34375     0.          0.          0.          0.          0.          0.\n",
      "   12.6640625   0.          0.          0.0859375   0.          0.4296875\n",
      "    0.          0.          3.0546875  20.5078125   0.          0.\n",
      "    1.2421875   0.          0.          3.2890625   0.          3.59375\n",
      "   14.2890625  11.9140625   0.          0.          9.1015625]]\n",
      "max 21.875\n",
      "min 0.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Testbench for a single ip layer\n",
    "net2 = {}\n",
    "net2['input_2'] = InputLayer((None, 50, 4, 4))\n",
    "net2['ip_2'] = DenseLayer(net2['input_2'], num_units=500, nonlinearity=rectify)\n",
    "weight = np.random.rand(800,500)*2-1\n",
    "#weight = np.arange(32)/32\n",
    "#weight = weight.reshape(8,4)\n",
    "#print('weight (caffe)',weight)\n",
    "\n",
    "net2['ip_2'].W.set_value(weight)\n",
    "input = np.random.rand(1, 50, 4, 4)\n",
    "#input = np.arange(8)/8\n",
    "#input = input.reshape(1,2,2,2)\n",
    "#print('input (caffe)',input)\n",
    "conv_output_test = lasagne.layers.get_output(net2['ip_2'], floatX(input), deterministic=True).eval()\n",
    "print(conv_output_test)\n",
    "print('max', np.amax(conv_output_test))\n",
    "print('min', np.amin(conv_output_test))\n",
    "print('##############################')\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(500, 50, 4, 4)\n",
    "#print('weight (fpga)',weight)\n",
    "#weight = weight[...,::-1,::-1]\n",
    "FPGALoadW(weight, 3, 4, 1, 0, flip_filters=False)\n",
    "#print('input (fpga)',input)\n",
    "FPGA_output = FPGAQuickTest(input, 1, 1, 500)\n",
    "print(FPGA_output.reshape(1,500))\n",
    "print('max', np.amax(FPGA_output))\n",
    "print('min', np.amin(FPGA_output))\n",
    "np.allclose(conv_output_test, FPGA_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make predictions on the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  81.97850518999999\n"
     ]
    }
   ],
   "source": [
    "start_time = time.process_time()\n",
    "prob = np.array(lasagne.layers.get_output(net['prob'], floatX(data_zeromean[0:500]), deterministic=True).eval())\n",
    "predicted = np.argmax(prob, 1)\n",
    "end_time = time.process_time()\n",
    "print(\"Elapsed Test Time: \", end_time-start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check our accuracy\n",
    "We expect around 75%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3 8 8 0 6 6 0 6 3 1 4 9 5 7 9 8 5 5 8 6 7 0 4 9 4 2 4 0 9 6 6 5 2 5 9 1 4\n",
      " 9 9 5 4 6 3 6 0 9 3 3 7 6 9 8 6 3 8 8 7 5 5 3 7 3 6 9 6 2 1 2 3 7 2 6 8 8\n",
      " 0 2 9 3 2 8 8 9 1 7 2 7 2 8 8 9 0 6 8 6 4 6 6 0 0 7 5 5 6 3 1 1 2 6 8 7 4\n",
      " 0 2 2 1 3 0 4 2 5 8 3 1 2 8 2 9 3 3 2 4 1 8 9 1 2 9 7 2 8 6 5 6 3 8 3 6 5\n",
      " 5 3 8 9 6 0 0 5 2 9 3 4 2 1 5 6 0 2 8 4 7 0 9 7 9 8 9 9 3 7 5 0 0 5 2 2 0\n",
      " 8 6 3 3 8 5 8 0 1 7 2 8 8 7 8 5 9 8 7 1 3 0 5 7 9 3 0 5 9 8 0 7 9 0 4 4 3\n",
      " 9 5 3 1 3 2 3 6 5 1 5 1 8 0 4 7 3 5 1 1 1 9 0 2 1 8 2 0 7 3 9 9 2 0 6 0 8\n",
      " 9 8 1 5 3 0 8 8 4 7 0 2 3 6 3 8 2 0 2 4 3 9 0 6 1 9 9 1 0 7 9 1 2 6 1 3 2\n",
      " 6 0 0 6 6 2 5 2 6 1 9 2 9 4 8 6 0 0 3 0 7 7 5 5 3 5 2 3 4 5 7 5 4 6 1 9 6\n",
      " 6 6 9 3 8 0 7 4 6 0 7 8 5 7 6 8 9 9 1 8 2 6 3 2 2 8 4 9 5 8 1 9 2 2 0 0 1\n",
      " 2 7 9 4 2 7 9 7 8 6 6 9 0 0 2 8 7 2 5 5 1 2 6 2 9 6 4 3 0 1 9 1 7 8 8 6 0\n",
      " 1 8 2 7 8 5 6 1 9 3 7 7 7 4 7 8 0 2 9 3 3 6 6 4 5 3 3 3 7 3 2 5 9 1 2 9 9\n",
      " 3 7 5 0 2 0 6 9 7 3 9 4 4 3 3 9 5 6 1 2 9 4 4 3 3 8 3 7 8 0 5 7 6 0 5 4 8\n",
      " 6 8 5 5 0 9 9 5 0 1 0 8 1 1 8 0 5 2 0]\n",
      "[3 8 8 0 6 6 1 6 3 1 0 9 5 7 9 8 5 7 8 6 7 0 4 9 5 2 4 0 9 6 6 5 4 5 9 2 4\n",
      " 1 9 5 4 6 5 6 0 9 3 9 7 6 9 8 0 3 8 8 7 7 4 6 7 3 6 3 6 2 1 2 3 7 2 6 8 8\n",
      " 0 2 9 3 3 8 8 1 1 7 2 5 2 7 8 9 0 3 8 6 4 6 6 0 0 7 4 5 6 3 1 1 3 6 8 7 4\n",
      " 0 6 2 1 3 0 4 2 7 8 3 1 2 8 0 8 3 5 2 4 1 8 9 1 2 9 7 2 9 6 5 6 3 8 7 6 2\n",
      " 5 2 8 9 6 0 0 5 2 9 5 4 2 1 6 6 8 4 8 4 5 0 9 9 9 8 9 9 3 7 5 0 0 5 2 2 3\n",
      " 8 6 3 4 0 5 8 0 1 7 2 8 8 7 8 5 1 8 7 1 3 0 5 7 9 7 4 5 9 8 0 7 9 8 2 7 6\n",
      " 9 4 3 9 6 4 7 6 5 1 5 8 8 0 4 0 5 5 1 1 8 9 0 3 1 9 2 2 5 3 9 9 4 0 3 0 0\n",
      " 9 8 1 5 7 0 8 2 4 7 0 2 3 6 3 8 5 0 3 4 3 9 0 6 1 0 9 1 0 7 9 1 2 6 9 3 4\n",
      " 6 0 0 6 6 6 3 2 6 1 8 2 1 6 8 6 8 0 4 0 7 7 5 5 3 5 2 3 4 1 7 5 4 6 1 9 3\n",
      " 6 6 9 3 8 0 7 2 6 2 5 8 5 4 6 8 9 9 1 0 2 2 7 3 2 8 0 9 5 8 1 9 4 1 3 8 1\n",
      " 4 7 9 4 2 7 0 7 0 6 6 9 0 9 2 8 7 2 2 5 1 2 6 2 9 6 2 3 0 3 9 8 7 8 8 4 0\n",
      " 1 8 2 7 9 3 6 1 9 0 7 3 7 4 5 0 0 2 9 3 4 0 6 2 5 3 7 3 7 2 5 3 1 1 4 9 9\n",
      " 5 7 5 0 2 2 2 9 7 3 9 4 3 5 4 6 5 6 1 4 3 4 4 3 7 8 3 7 8 0 5 7 6 0 5 4 8\n",
      " 6 8 5 5 9 9 9 5 0 1 0 8 1 1 8 0 2 2 0]\n",
      "0.746\n"
     ]
    }
   ],
   "source": [
    "accuracy = np.mean(predicted == data['labels'][0:500])\n",
    "print(predicted)\n",
    "print(data['labels'][0:500])\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[199, 450, 600, 937, 1571, 1576, 1597, 1625, 1651, 1783, 3246, 3326, 3527, 4015, 4664, 4679, 4729, 4770, 4922]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5sAAABuCAYAAACp3AcnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VNX5B/DvNJAARTaJIUCUpYRACCFhawXhh4KgtFIQ\nEKQgEbUqWouoqNVSazUoimLFqlgXrGwqigtYFlFZFIwBkV1lXxXBIAmQBO7vj9tz8o6ZkExy59yZ\nO9/P8/DwepLMHN8Md+6d8973+CzLAhEREREREZGTfuH2BIiIiIiIiMh7QnKxWbNmzYM+n8+qzJ+a\nNWseDMWcvKqyuWaeg8dcm8Ncm8FjtTnMtTk8fpjDXJvDXJvBY7XzfKEoo/X5fFZlH9fn88GyLJ/D\nU/KsyuaaeQ4ec20Oc20Gj9XmMNfm8PhhDnNtDnNtBo/VzmMZLRERERERETmOF5tERERERETkOF5s\nEhERERERkeOMXmzm5eVhyJAhaNOmDVJTU7F69WqTTx9VPvjgA6SkpCA5ORmPPPKI29PxrG3btiEj\nIwOZmZnIyMhA3bp18dRTT7k9LU9q1qwZ0tPTkZGRgS5durg9HU+bOnUq0tLSkJaWxtdzCPH4YRbf\nF83Izs5Gamoq2rdvjxEjRqCwsNDtKXkWX9PmjBkzBgkJCWjfvr3bU4k8lmU5/sd+2NKuueYa68UX\nX7Qsy7KKioqsvLy8Ut/zv58Nyby8+CdQrk+fPm21bNnS2rlzp1VYWGilp6dbmzdvZp5DkOuf5z0x\nMdHavXs3cx2CXDdv3tw6cuTIWX4DzLUTed6wYYOVlpZmnTx50iouLrb69Oljffvtt8x1CHItlXX8\nYK6dyTXfF83keufOnVbz5s2tU6dOWZZlWUOHDrVeeeWVUr8P5rrqua7Ia5q5rnqeleXLl1tr1661\n0tLSAn6duS77j7GVzWPHjmH58uXIysoCAFSrVg116tQx9fRRZc2aNWjVqhUuuOACVK9eHcOGDcP8\n+fPdnpbnLVmyBC1btkRSUpLbU/Eky7Jw5swZt6fheZs3b0bXrl0RFxeHmJgY9OjRA/PmzXN7Wp7H\n40do8X3RjDp16iA2Nhb5+fkoLi5GQUEBGjdu7Pa0PImvabO6d++O+vXruz2NiGTsYnPHjh1o2LAh\nsrKykJmZiRtuuAEnTpww9fRRZd++fX4nLE2bNsW+fftcnFF0mDNnDoYPH+72NDzL5/OhT58+6Ny5\nM6ZPn+72dDyrXbt2WL58OY4ePYqCggIsWLAAe/bscXtansfjR2jxfdGM+vXrY/z48Tj//PPRpEkT\n1KtXD71793Z7Wp7E1zRFCmMXm8XFxcjNzcXYsWORm5uLWrVqYdKkSaaeniikioqK8M4772DIkCFu\nT8WzVq5cidzcXCxYsADTpk3DihUr3J6SJ6WkpGDChAno06cPLr/8cmRkZCAmJsbtaXkajx/kFdu3\nb8cTTzyBXbt2Yf/+/Th+/Dhmzpzp9rSIyEXGLjabNm2KpKQkdOrUCQAwePBg5Obmmnr6qNKkSRPs\n3r1b//fevXvRpEkTF2fkfQsXLkTHjh0RHx/v9lQ8KzExEQAQHx+PgQMHYs2aNS7PyLuysrKQk5OD\njz76CPXq1UNycrLbU/I0Hj9Cj++LZuTk5KBbt25o0KABYmJiMGjQIKxatcrtaXkSX9MUKaqZeqKE\nhAQkJSVh27ZtSE5OxtKlS9G2bduA3+vz+SxT8wpHlmX5qvLznTt3xjfffINdu3YhMTERs2fPxqxZ\ns0p9H/NctTxLs2bNOmsJHHNdtVwXFBTgzJkzqF27NvLz87Fo0SJMnDgx4Pcy11V/XX///feIj4/H\n7t278dZbb+Gzzz4L+H3MtTPHkPKOHwBzzfdFM6qa59atW+PBBx/EyZMnERcXh6VLl6Jz584Bv5e5\nNvOaBphrp47VoolQmZjrALkORdchlNHJad26dVanTp2s9PR0a+DAgdaPP/5Y6nsARP0fJ3K9cOFC\nKzk52frVr35lZWdnM89VzPPZcp2fn281bNjQOnbsWMCvu/3/GQ5/qprr7du3W+np6VaHDh2sdu3a\nBXxNM9fB5/rneVYuuugiKzU11erQoYO1bNky5jqEuS7v+MFcO5drvi86m+eycv3oo49abdu2tdLS\n0qxRo0ZZhYWFzHWIcl3ea5q5Dj7XgfJsWZY1fPhwKzEx0YqNjbWSkpL07hrMdfm59v0vsY7y+XxW\nZR/X53NssSliWUF8AlPZXDPPweUZYK6rgrk2x8Tx438/W6mf8xLm2hy+L5rBY7U5zLU5PFabEyjX\nxu7ZJCIiIiIioujBi00iIiIiIiJynLEGQaalpqbqeO3atTp+7LHHAAD33nuv8TkRERERERFFC65s\nEhERERERkeNCsrJZo0aNQz6fL6EyPxsXF4dTp05VeQ4tW7bUsdyQPBQNkdxU2Vw7ledowlybw1yb\nEQ7H6mjBXJvD44c5zLU5zLUZPFY7LyQXmydOnGik4mD3m+EvKTiVzTXzHDzm2hyVax4/QovHanOY\na3N4rDaHuTaH74tm8FjtPJbREhERERERkeM82yDo17/+dcDxxYsXG54JERFR6J1zzjk63rp1KwDg\nzTff1GNPPPGEjrdv316l5xo1apSOZ8yYUaXHIiIiZz3++OMAgD//+c967Be/KFljVA1TAeDOO+8M\n6Vy4sklERERERESO48UmEREREREROc6zZbSynEgqLi42PBMi8poHHnhAx/fff7+OfT6fjg8ePAgA\n6NWrlx7bsmWLgdlRtEpLS9NxXl4eACAlJUWP1alT56w/37RpUx0fOnRIx0VFRaW+l6WzZErPnj0B\nAB999JEe27Fjh467deum4wMHDhibF1G4GTdunI5V+eyZM2cCfq8sr2UZLREREREREUUcXmwSERER\nERGR4zxbRtujRw8dFxQU6Pjbb791YzpEjkhOTtbxv//9bwD+JUSyjFN1i3zrrbf02PHjx0M9xYjT\nvn17AEC/fv302K233qrj9evX6/iSSy4BAFSrVnLotCwrYHzeeecBAF5//XU9JssciZy2atUqHauO\n7LKEKj09/aw//+OPP+r49OnTZ/3em266SceLFi3ScbS9x/7yl7/UsSxDvv7663WscpWTk6PH4uPj\nddymTRsAwKZNm/TYwoULdfzoo4/q+LvvvnNi2hElMzMTgP9r+dixYzrm+5q3de/eXcetW7fW8Rdf\nfKHjL7/8EoD/e3C0UF1nAf/SWHU+KDvQynPE4cOHG5idjSubRERERERE5DhPrWzKTz/kCsKSJUt0\nzJvHK2/OnDk6lp8erV69WsdyH7eKkjc0S5V5LC+KjY3V8SOPPKLjCy+8EEDZK2svv/wyACAjI0OP\n3X777aGaZsRSq5jXXnttwK83bty4So8fExNTpZ+PdtWrV9ex/FR78uTJOu7bty8AYNq0aXpMrk5H\nI9UgSFqxYsVZf6YiK0T169cHAHz44Yd6LNpWM4GS1+Wzzz6rxxITE3V88cUXl/oZWXEV6LitVjh/\nHo8dO1bHvXv3BgCsXLmy0nP3gnbt2ul44MCBOmbjqsgmz+NVY6h7771Xj9WsWTPgz6n3b3Xe43WB\nGgEB/qv/akUz0BhgdhWYK5tERERERETkOF5sEhERERERkeM8VUarltwB/+Xh6dOnuzEdz1CNJoYO\nHarH5LL8VVddpWNVcjh+/Hg9Jn8X6ubkQGM/H1c3Pcubn0O9F1A4Uk1pAOCKK64o9XXZMKJu3bo6\njouLAwDccsstekw28vjggw8cnWckycrK0vHo0aND+lxr1qwJ6eN7VWpqKgBgypQpeqxPnz463rp1\nq47V/nuFhYVmJhfFBg0aBAB45pln9Jg8RpVXqusV5557LgD/sr/atWvreMOGDTr+4YcfAPgff1UD\nMQBo27YtAP8yXFkmWqNGDR3fddddAIABAwZU7X8gguTm5p7167fddpuOVUO8n376ybHn79ixo45l\nUxqqPLnnr2pmCADZ2dk6Vo34vvrqKz328ccf67h58+Y6vvLKKwF4v4w2KSkJADB48GA9Js+hAzUD\nkmN33HGHjt98882QzfPnuLJJREREREREjuPFJhERERERETnOE2W0TZo0AQDceOONemzZsmU6fvvt\nt43PKdLJTlddu3YF4F86K2NJdcWS5bDBdMcKNC47be3fv1/H0dKtVnZik9Q+eLLER3UqBIAXX3wR\ngH831L/85S86juYy2p07d+r45MmTAIBatWpV+XEPHz6s4+effx4A8PDDD1f5cb1MlSMCJaVQAPDg\ngw8CKNk/DSjZExUAvv/+ex2rDoXy90pVI/ePfOGFF3SsurAOGzZMj0VjaaEqjZ07d64eU+WwALBv\n3z4dT5w4EQBw6NChsz6m6vQLAPPnz9fxRRddpGNVSq5ubwGAzz77LKi5RxpVOilLh//+97/rWOZa\n7RVbr149PVbZktoOHToA8L8t6Oqrr67UY5Ht7rvvBgDcfPPNekzuT7t9+3Ydq7zLfwuS7E5ucs9I\nN6l/9126dNFj5Z1vy1vR3Dpv5somEREREREROY4Xm0REREREROQ4T5TRqo2SVTktADz99NM6Lioq\nMj6nSCTLcmQHSLUsL8tdpUDjwXTHkt8ry4G6detW6ntlOYDqfOvFDrXJyck6ll0JJfV7kaXFckNr\n1YVZdXQD/P+NRDNZZv/JJ58AAPr161flx5XlWs899xwA4MSJE1V+XK8ZMmSIjq+//nody5Lwm266\nCYB/xzxVOg74dzOMjY0NyTyjmTzuqmOt9P777+s4GrsA33fffQCAli1b6rHNmzfrOCcnR8f5+fkV\nesyjR4/q+MCBAwG/R3WmnTBhgh4bOHBghR4/0r333ns6/vTTT3X87LPP6liV0quutID/7SXlke+9\nqqx5zJgxQc+VSshbSVR3fNm5WZZ2Tpo0ScfyVolAZEfy5cuXV3me4Uqem8+ZMwdA2Ts6BDq3DoeO\n+FzZJCIiIiIiIsd5YmVT7nOlLF261IWZRDbZiKe8ZkBlNQgKtAoqVyPVJyxlfSojVzbV3llyXvJx\n1bgXVzblp+VyBYecN2/ePADOrGzKfb/UJ+tqX0IA2LNnT5WfI5K98cYbAEoabwD+ef/mm28q/Fiy\nWZBqMCH3a5PHjW3btgU/2Sinmi4BJdVDALBjxw4A/sdtL5Or7bIJiWpm1b9/fz22adMmx55X7R0L\nAI0aNdKx+l3IBk7RSDVoAvwrJV599VUA/jkLhmpMBgC9evUC4F9dxEqVsslz8YceekjH8vihzv2O\nHTumx+Q5YnmrmWXx8jm/fC9T+atIk011Ph0ODcS4sklERERERESO48UmEREREREROS5iy2jl3mxq\nv68VK1boMXlzPp2dKmGQpSiBmv6U1SBIlcYBJTd6O7FsH6hBUFk3Qkej1157ze0peIIqd/3b3/6m\nxwI1RAlWZmYmgJIyXcC/kcfevXur/ByR4NZbb9WxKqMfOXKkHgumLC0xMVHHffv21bEqo5Uluer4\nAbCMtjLkPm6S2p/31KlTJqfjGtm0R+0xCviXzIeCbIwlb6tQe27KW1GoRG5ubqkx2YRMfV3uDduz\nZ08dt2rVSseqMZAs2SVb3bp1dXzZZZcBAP71r3/pMfn+JkuTVcNCeXtJtBxLgpGUlBQwDqbJptyf\n123RfbZOREREREREIcGLTSIiIiIiInJcxJbRXnzxxTquV68eAP9yTqo41emqvA6zkuwedtddd4Vk\nXkOHDgUAzJo1S49deOGFpeal9h0C/PdrCocOXJV15MgRHcsSk5MnT+p45syZRufkVSrXS5Ys0WOq\n0yTg3/Vx1apVAPw76clywwYNGpR6fFVOCwDXXXedjmXZrpe99NJLOj5+/HiVHkvuPXj//ffreMCA\nAQD8b6VQvyuqHJlf+Xt78skn3ZiOa1q0aKHjNm3aGHte2Vn59ddf1/Htt99ubA6RSJ0DnHPOOXrs\nggsu0HFeXh4A/+Ov6nwP+Jcnq/LQd999NyRzjWRz587VcZ8+fQAA3333nR6Te0PKve6//vprAP5l\ntixTLm327Nk6lucY5XWjDdfjM1c2iYiIiIiIyHG82CQiIiIiIiLHRVQZbWxsrI7/+te/6ri4uBiA\nfyesUKlTp46OZfdW1WFLbSgMAAUFBSGfjxP27dsHwL/sTHX4BYBx48aV+hlZrhoqqsxCdtQKVDog\nfw+DBw/WcUxMTKinGDKrV6/W8aJFi3QsSzoDbWKcmpqq40DdessqlSYgKytLx1OnTtVxrVq1dLx+\n/XoA/mWFcsPvsWPH6lh122vYsKEekyXnsnTon//8Z5XmHs4qWzqryuDkLRNyo/aDBw+WGt+4caMe\n27p1a6WeN5qlpKToWJYTylsVoq2zr+yWLF9zJt18882lxuRrnUr76aefdHzVVVfp+J577in1vV99\n9ZWO5bnNjBkzQjS7yCHPed977z0dy27fkydPBuBfel9YWKjjHj166Fh1Vp0+fbrzk41wsvT4N7/5\njY7lsbi8brQyDidc2SQiIiIiIiLHRdTKpmwO07ZtWx2rT7DlTchOuvTSS3X89ttv67hGjRo6Vp88\nyJWlP/3pTyGZj9PK26/LxCpmRcm5qlU6+QmPF1fufv/731f4e+UnY4FWNvlpYsWsW7euwt+7YcMG\nHd900006Vvtryv1Q5f7AcpXTyyubgcTFxelYNsCS+29u3rwZgP8K8Pz583UsGy+9/PLLAIBevXo5\nPtdosmXLFh0nJyfrODs7243phIWVK1cGjE3Kz88vNbZw4UIXZhI5EhISdPzHP/7xrN8rq4S+/PLL\nkM0p3MlVsa5duwLwP2eQ593//e9/dTxx4kQA/quZsrJMVsepXD///PNOTdszVLNOIPC5LlByXifH\nZDPMYBpjmqxa5MomEREREREROY4Xm0REREREROS4iCqjHTNmTMDx1q1bAwAuueQSPRaoeUqw7rvv\nPgD++1rJ8q9A6tevX+XnNe38888HAIwfP97lmfhTjX9kaYEs81DlBHfccYce+/TTTw3NLjzJBh/k\nrsWLFwMAli1bpsdkA6toIfe769evHwD/5iabNm3SsWyGpRqzyD1NpdGjR+tY7UMYTPkzlVC/I9lI\nRZWBA/7ly9FG7neZlpam4zvvvNPx55LNxGRjls8//1zH6rYetV8hBfbCCy/oOND+x5IsyW/WrJmO\nb731VgD++/t6jbzdRu4v/cwzzwAo2cceKH9/9erVqwf83r59++pYNbuSe3KSraxGP4GaAcmx/fv3\n61juXxpOuLJJREREREREjuPFJhERERERETkuospob7jhBh3LzrTNmzcHAFxzzTV6LCcnR8d5eXkV\nfo5OnTrpWO3lKffWOnLkiI5btmyp4+3btwMoKb2NJKqrVXldaUNFdsSSnbBmz54NwL/rVqAOXbJ0\nNphOXF7Us2fPUmOyDNHEXrRkU+VY8lgVjeR+d6rctSyHDh0669dlt+8LLrhAx1OmTKnk7Ago6TIp\nO0+2atXKremEFVnOOmvWrJA+l7z9R5bsyn9DGRkZAICdO3eGdC6Rrn///jqW5w2q26x8r5T5/e1v\nf2tgduFDdu2V++mq/ZFHjBihx+bOnRvwMdTtZXLP81tuuUXH06ZN07HqHE4l1C4CqgMwEFw3Wvm9\n8nxaPa78utzD3eROE1zZJCIiIiIiIsfxYpOIiIiIiIgcF1FltKo7IeDfNevdd98FAPzhD3/QY717\n99bx5MmTday6usnubrLr0z/+8Q8dq01pY2Nj9dh5552nY1mqq7r47dq1q8L/P+FCbr7rBtm1bM+e\nPToO1HXrjTfe0LEqAYj20tnyLFiwQMdldfYkZ9SsWVPHqpylcePGbk3Hc2RpVtOmTXX88ccfuzGd\niJaenn7Wrx8+fNjQTMKbLO2W70VO6tixI4CSW4J+7sEHH9SxumWHAlNlsE8//bQeU6XHQEnJvSyd\nld57770Qzi48VKtWcuovS2NV6SxQUj6rzq/PRnWYledy8nFVV18KLCkpye9vILhutEOHDtWxLGVW\n3yu/Ls+hTeLKJhERERERETkuolY2JbmfWnJyMgD//RgHDRqk4+zsbB2rfYDkp1ryE4TatWvruKio\nCADwxRdf6DH5qcB//vOfUt8bidQnUPKG42HDhhl7fnnz8m233VZqXM5LrmJyRdMmV3hkTGbUqFFD\nx5MmTdKxXIUL5OjRoyGbk5cMHz5cxzfeeKOOL7/8cjem4xlyP0G1z6bcu7G4uNj0lMKKyo9c7ZHV\nTFUlm12p/WdlZdQnn3yi48cee8yx5/Uiuce6ahT50EMP6TG5Sh+u+xCaJFe/unXrpmO5J3SgFU21\nAg/47zOr9n6Vr9O7777bmclGAXXtEqgRUFnjFfledY4cDufKXNkkIiIiIiIix/Fik4iIiIiIiBzn\nC/Xeij6fz53NG4XWrVvrOD4+vtTX5X5W8qbymTNnAgBGjhwZwtmVZlmWr/zvKq2yuVZ78cg9lvbt\n26fjQI141M8AgffOk2Ua5ZHPK1+PqmRZ7gXkZDlAZfMMhMfrWpF7w8o9lBS1Xyngv2eWSaZf08Ho\n3r07AGD06NF6bOPGjToOtLeYbOQxYcIEHZdX2nn69Gkdy+OK/DdQVeGc62CofTTlrQsFBQU6DrSn\nrGmRnOsBAwboWO0fedlll+mxcGu6FMm5VuTej3LPTlWyvGTJEj127bXX6liW8oZaJL4v1qpVS8cN\nGjQAADz66KN6bMyYMTqWjSbLo8pG5a1UTnLrNb1582YdJyYm6lieC6vjr7y1Se5BKhtnZWVlAfAv\nww034Xb8kO/56nxZnv/K2/sCjcsxeb4umwG5VT4bKNdc2SQiIiIiIiLH8WKTiIiIiIiIHBex3WiD\nsXXr1oCxIpea27Ztq2NVcnjffffpMbkPp1eo/3+1VygArFy5UseqnHjNmjV6rEuXLjoePHiwjtXS\nviyjLa+TliwHkPs0HThwwG9+FFh5Zd6vvvqqoZlEpvHjxwMArrjiioBfd7IT5LRp03TsZOmsF2Vm\nZgIAzj33XD0WTHk+laY6twPAHXfcoeMNGzYA8O+ASs574IEHdCz35P3hhx8AAO+//74eM1k6G+lk\nF3a104Da+xHwL0kMhjq3+fDDD/VY3bp1K/VY4UTeTiZ3YFixYoWO1Z6P+fn5ekze0nDXXXfpWJ2r\nUcXJ895AOy+U141WnivPmzdPx+F6vsyVTSIiIiIiInIcLzaJiIiIiIjIcVHRjTbShEPXrFWrVulY\nddCUm/+W1zUrmE5aspxLdp4NtUjsuidVq2ZXwW/btk2PqQ5yUq9evXTsVplcOLymy7Jnzx4AQOPG\njR19XFX6Ijdql938tmzZ4ujzKeGc6/LExcXpWHVW/vrrr/VYuJXRRkKu1YbhAPDjjz/qeOrUqTpW\nr8WuXbuamlbQTOe6RYsWAPxLBIPpZCqlpqYCKClXBoAvv/xSxx06dKjU44ZCpL8vDhs2DABQr149\nPXb48GEdr1+/HoB/mbIqYwaA6667Tseq+7W8verOO+90bK5uHT/k6+3JJ5/Usfz3v2jRIgDAlClT\n9Fi4dagORrgdq8eNG6dj9b4m81/WObQ6XzZ5rhwsdqMlIiIiIiIiI7iyGYbC4RMYecO9IvfWlHsv\nXXjhhTpWqznB3NwsbzQ3KdI/wY2NjQVQ/qftXNk8O7XyGOg1Hyx5PJ08eTIA4J577qny4wY5h7DN\ndXnkyrxaTVi7dq0eO3jwoPE5nU0k5Fr++5eNTgoLC3WsVtwXL15salpBM53r7OxsAMBzzz0X8Os7\nd+4EADRr1kyPyWZWF110kY4nTZoEwP9YLfc0DaemHpH+vqjI34XcJ1PtySlXNo8cOaJj+TtSq5hy\ndVtWqlRVOBw/5LnaOeeco+O8vDynniIshEOuy6LOPeTeu/K8OhzOl4PBlU0iIiIiIiIyghebRERE\nRERE5DiW0YahcF7u95JILxdS5S9Lly7VYz169NCxKj/s37+/Hjt06JCh2fkL59e0apYg8ySlpKTo\nOCEhAQDQvHlzPfbyyy/reP/+/Tp+6aWXnJxmhYVzrgPp3r27jmXp2/z5892YTlAiLdeRzK1c/+53\nv9OxLKlVTWfkOZQsSVRNgQBg48aNAPz37H3llVeqMq2QifT3xUBGjRql47S0NABAx44d9VirVq10\nLPeirWxDqIri8cMc5tocltESERERERGREbzYJCIiIiIiIsdVc3sCRFQ5qsuv6nQI+JfRqjItt0pn\nI8W6dev8/qbQkx085X5j4bx3GEUnuV9jo0aNdJyYmAgA+O677/TYpk2bdKzKNYGSDqayAyqZM2PG\nDLenQBTVuLJJREREREREjuPFJhERERERETmO3WjDELtmmeHFrnvhiq9pcyIh1926ddPx3Llzddyk\nSRNTU3BEJOTaK5hrM/i+aA5f0+Yw1+awGy0REREREREZwZXNMMRPYMzgJ7jm8DVtDnNtDnNtDnNt\nBt8XzeFr2hzm2hyubBIREREREZERvNgkIiIiIiIix4Vkn82aNWsePHnyZEJlfrZGjRqHTpw40aj8\n7ySg8rlmnoPHXJvDXJvBY7U5zLU5PH6Yw1ybw1ybwWO180Jyz6bP57Mq+7g+n69K9wxEm8rmmnkO\nHnNtDnNtBo/V5jDX5vD4YQ5zbQ5zbQaP1c5jGS0RERERERE5jhebRERERERE5DhebBIREREREZHj\nzF9snjkDZGYCV1xh/KmjSrNmQHo6kJEBdOni9my8bepUIC3N/vPUU27PxptOnQK6drVfz6mpwL33\nuj0j79q2zc5zZqb9d926fF2HypgxQEIC0L692zPxPr6uzdm7F7j4YvtYzffF0MrOtvPcvj0wYgRQ\nWOj2jLwrLw8YMgRo08bO+erVbs8oYphvEPTEE8AXXwDHjgHvvBPoZ3lzbRDKzHWLFnae69cv6+eY\n5yAFzPXGjcDw4cDnnwPVqgGXXQY8+6yd/5KfY66DFDDXBQVArVrA6dNAt27A44/bf/v/HHMdhHIb\nIZw5AzRtar+pJiX9/GeZ6yAEzPWKFUDt2sCoUcD69Wf7WeY6CJV9XTPPwQuY64MH7T8dOgDHjwMd\nOwLz5wPG18fNAAAEBElEQVQpKfLnmOsglcr1rl1Ar17Ali1AbCxw1VVA//728cT/55jrIJR5/Bg9\nGujZE8jKAoqL7XOSOnV+/rPMdQBmVzb37gUWLACuu87o00Yly7LfUCm0Nm+2V9zi4oCYGKBHD2De\nPLdn5U21atl/nzplv7bL+CCFHLRkCdCyZakLTXJI9+58HbuBr+vQatTIvtAE7A9T2rQB9u1zd05e\nVKeOfZGZn19y8dO4sduz8qZjx4Dly+0LTcBeXPjZhSaVzezF5rhxwOTJgI8X/SHn8wF9+gCdOwPT\np7s9G+9q184+AB09ah/oFywA9uxxe1bedOaMXf7WqBHwf/8HtG3r9oy8b84ce+WeyEv4ujZn505g\n3Tr7Q1lyVv36wPjxwPnnA02aAPXqAb17uz0rb9qxA2jY0L7YzMwEbrgBOHHC7VlFDHMXm++/b9+b\n0qGDveoWgvJdElauBHJz7YufadPsci1yXkoKMGGCfWF/+eX2xVBMjNuz8qZf/AJYu9aukPjkE+Dj\nj92ekbcVFdm3OgwZ4vZMiJzD17U5x48DgwfbfQ1q13Z7Nt6zfbt9a9quXcD+/Xa+Z850e1beVFxs\nn1OPHWv/XasWMGmS27OKGOYuNleutA/wLVrYnyguW1aqrpwclJho/x0fDwwcCKxZ4+58vCwrC8jJ\nAT76yP5kMTnZ7Rl5W5069n0pOTluz8TbFi6077WKj3d7JkTO4evajOJi+0Jz5EhgwAC3Z+NNOTl2\n34IGDewPuQcNAlatcntW3tS0qV1236mT/d+DB9sXnVQh5i42H34Y2L3b/iRm9my7U9mMGcaePqoU\nFNifcAF2Lf+iRXa5J4XG99/bf+/eDbz1FnD11e7Ox4sOH7Y7wQF26crixSX3BFFozJrFUkMTWOlj\nFl/XZlx7rX2rw223uT0T72rdGvjsM+DkSfsYsnSpfX8sOS8hwb7Y3LbN/u+lS3krTxCquT0BCoFD\nh+zVTJ/P/nRxxAjg0kvdnpV3XXklcOQIUL068MwzvGk8FA4cAK65pqTx1ciRwCWXuD0r7yoosJuo\nPP+82zPxtquvtisifvjBvu/qgQdKGlCQ8/i6NmPlSuC11+xtTzIy7HORhx8G+vVze2bekp5uVwh2\n7GivbGZk2PcSUmg89ZR9Pl1UZFdpvvSS2zOKGOa3Pin/Z9k2OAiVzTXzHDzm2hzm2gweq81hrs3h\n8cMc5toc5toMHqudZ7YbLREREREREUUFXmwSERERERGR43ixSURERERERI7jxSYRERERERE5LiTd\naGvUqHHI5/MlVPZnnZ6Pl1U218xz8Jhrc5hrM3isNoe5NofHD3OYa3OYazN4rHZeSLrREhERERER\nUXRjGS0RERERERE57v8BON9LOWu0IagAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2746de50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wrong = predicted != test_label[0][5000:10000]\n",
    "wrong_ind = [i for i,x in enumerate(wrong) if x == 1]\n",
    "print(wrong_ind)\n",
    "\n",
    "def make_image(X):\n",
    "    im = np.swapaxes(X.T, 0, 1)\n",
    "    im = im - im.min()\n",
    "    im = im * 1.0 / im.max()\n",
    "    return im\n",
    "\n",
    "plt.figure(figsize=(16, 5))\n",
    "for i in range(0, 10):\n",
    "    plt.subplot(1, 10, i+1)\n",
    "    plt.imshow(make_image(test_data[wrong_ind[i]+5000][0]), interpolation='nearest', cmap=plt.get_cmap('gray'))\n",
    "    true = test_label[0][wrong_ind[i]+5000]\n",
    "    pred = predicted[wrong_ind[i]]\n",
    "    color = 'green' if true == pred else 'red'\n",
    "    plt.text(0, 0, true, color='black', bbox=dict(facecolor='white', alpha=1))\n",
    "    plt.text(0, 32, pred, color=color, bbox=dict(facecolor='white', alpha=1))\n",
    "\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Double check\n",
    "Let's compare predictions against Caffe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net_caffe.blobs['data'].reshape(500, 3, 32, 32)\n",
    "net_caffe.blobs['data'].data[:] = data['raw'][0:500]\n",
    "prob_caffe = net_caffe.forward()['prob']#[:,:,0,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose(prob, prob_caffe)\n",
    "# print(prob)\n",
    "# print(prob_caffe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph some images and predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5sAAABuCAYAAACp3AcnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX6P/DPEBAikaahFylGAgkkIEU2soo0qUZhDSAq\nTXYVRWzo6rrYsKBo6GUtgEhRUZDqokQhwvIDgy4BzA8QQpCqSAtSzHz/YM/JM8yEJDN3zs3c+bz3\nxYvHM5m5z55c7syd+9znuNxuN4iIiIiIiIisVMruBIiIiIiIiMh5gnKyGRkZedDlcrn9+RMZGXkw\nGDk5lb9zzXkuPs61OZxrM3isNodzbQ6PH+Zwrs3hXJvBY7X1XMEoo3W5XG5/X9flcsHtdrssTsmx\n/J1rznPxca7N4VybwWO1OZxrc3j8MIdzbQ7n2gweq63HMloiIiIiIiKyHE82iYiIiIiIyHI82SQi\nIiIiIiLLGTvZzMrKQmJiIlq0aIHExERUrFgREyZMMLX5sJKTk4MOHTqgadOmiI+P5zwH0ZAhQ1Ct\nWjU0a9bM7lQcb+XKlWjcuDFiYmLw2muv2Z2O4+Xl5aFFixbo1auX3ak4Fo8fZnGfNiM1NRXx8fH8\n/GHAtddei+bNmyMxMRGtW7e2Ox1H437tP2MnmzExMcjIyMB3332HzZs3o3z58khOTja1+bBSunRp\njB8/HpmZmVi/fj0mT56MHTt22J2WIw0aNAirVq2yOw3Hy8vLw4gRI7Bq1SpkZmZi3rx53KeDLDU1\nFU2aNLE7DUfj8cMs7tPBl5mZiXfeeQebNm3Cli1bsHTpUuzevdvutByrVKlSSEtLQ0ZGBjZu3Gh3\nOo7F/TowtpTRrl69Gg0bNkSdOnXs2LzjVa9eHQkJCQCAqKgoxMbGYv/+/TZn5UxJSUmoXLmy3Wk4\n3saNG3HdddehXr16KFOmDFJSUrB48WK703KsnJwcLF++HEOHDrU7FUfj8cMc7tNmbN++HW3atEHZ\nsmURERGB9u3bY9GiRXan5Vhutxt5eXl2p+F43K8DY8vJ5oIFC9CvXz87Nh129uzZgy1btqBNmzZ2\np0Lkt/3793t8OVW7dm1+gRJEo0aNwrhx4+BysYM7OQP3aTPi4uKwdu1aHDt2DLm5uVi+fDn27dtn\nd1qO5XK50KlTJ7Rq1QozZ860Ox3H4n4dmNKmN3j+/HksWbIEr776qulNh51Tp06hT58+SE1NRVRU\nlN3pEFEIWLZsGapVq4aEhASkpaUhGGsxE5nEfdqcxo0bY/To0ejUqROioqKQmJiIiIgIu9NyrPT0\ndNSoUQNHjhxBp06dEBsbi6SkJLvTchzu14ExfmVzxYoVaNmyJaKjo01vOqxcuHABffr0wcCBA9G7\nd2+70yEKSK1atZCdna3/OycnB7Vq1bIxI+dKT0/HkiVL0KBBA/Tr1w9r1qzBPffcY3daRH7jPm3W\noEGDsGnTJqSlpaFSpUqIiYmxOyXHqlGjBgAgOjoaycnJvG8ziLhfB8Dtdlv+5+LL+paSkuJ+//33\nC3wcQNj/sWKuBw4c6B41ahTn2aJ5vtxc//TTT+64uDjOdRDn+sKFC+6GDRu69+zZ4z579qy7efPm\n7m3btnGuA5zrS+f5Umlpae6ePXtyvw7iXBd2/OBcW7tfc5+2bp4LmuvDhw+73W63e+/eve7Y2Fj3\n8ePHOddBmOvTp0+7T5486Xa73e5Tp06527Vr5161ahXnOsC5vnSeuV8HPtdGr2zm5uZi9erVuOOO\nO0xuNuykp6dj7ty5+Oqrr/RyMytXrrQ7LUfq378/2rVrh6ysLNStWxfvvfee3Sk5UkREBCZNmoTO\nnTujadOmSElJQWxsrN1pEQWExw9yojvvvBNxcXHo3bs3pkyZggoVKtidkiMdOnQISUlJSExMRNu2\nbdGzZ0907tzZ7rQci/u1/1z/O4u39kVdLre/r8ub9wG3213kSfB3rjnPxZtngHMdCM61OSaOH/97\nrl/PcxLOtTl8XzSDx2pzONfm8Fhtjq+5tqUbLRERERERETkbTzaJiIiIiIjIcjzZJCIiIiIiIsvx\nZJOIiIiIiIgsVzoYL1quXLlDLpermj/PLVu2LM6ePWt1So7l71wHOs+PP/64jiMjI3XcrFkzAECf\nPn18Pm/q1KkAgPXr1+uxOXPm+J2HSXbNdTjiXJvBY7U5nGtzePwwh3NtDufaDB6rrReUk80zZ85U\nV7HL5SpWSyf+korH37nmPBcf59ocNdc8fgQXj9XmcK7N4bHaHM61OXxfNIPHauuxjJaIiIiIiIgs\nF5R1Nj02UMxvBaj4ay8pwZ7rBQsW6LigMtmi2rVrl447duyo4+zs7IBetzj8nWcgtPbrmJgYHe/Y\nsUPHI0eO1PHEiRODmkNJ3aeLonz58joeN24cAGD48OF6bPPmzTru27evjvfu3WsgO2+hPNehhnNt\nDufajHB5XywJwnGfrly5MgCgbt26hf6seg8dNWqUHtu6dauOs7KydPz9999f9rXCca7twnU2iYiI\niIiIyAiebBIREREREZHlgtIgiJxFlc8WpXRWlWmuWrVKjzVo0EDHPXv2BAA0bNhQjw0YMEDHr7zy\nSmDJkpfExEQd5+Xl6TgnJ8eOdEJOjRo1dDxs2DAAnvPYsmVLHffo0UPHkydPNpBdaGrRooWOFy1a\npONrr73Wsm107txZx9u3b9fxvn37LNtGuFDHbQBYsmQJAGDEiBF6bNq0aTr+448/zCVWAlStWlXH\nCxcu1PG3336r4xkzZgAA9uzZE5QcKlasqOP27dsDAFauXKnHzp8/H5TtEhWke/fuOu7Vq5eOb775\nZgBAo0aNCn0NVSZbr149PVa2bFmfPxsREeFPmmQIr2wSERERERGR5XiySURERERERJZjGS35dMMN\nN+g4OTnZ6/HMzEwdyxKJo0ePAgBOnTqlx6644godb9iwAQDQvHlzPXb11VdbkDEVJCEhQcenT5/W\n8aeffmpHOiEhOjpax7NmzbIxE2fq0qWLjgsqiwqULP0cPHiwjlNSUoKyPaeRx+UpU6Z4PT5p0iQd\nv/vuuzo+c+ZMcBMrIVRXTfleKMtZDx06pONglM/KbcmO2OrYJcv7d+7cafn2S5IKFSroWN2KExcX\np8dkx3uWFFtD3gr14IMPAsi/zQQAIiMjdexy+dfgWHbSp9DGK5tERERERERkuRJ/ZVM2pZHfmvz8\n8886/v333wEAc+fO1WMHDx7UsdO/1QsG2RRFfSslv8GVVyYOHDhw2dd67LHHdNykSROvx5ctW+Z3\nnlQw9c2ubOQxZ84cu9Ip8R5++GEd33777Tpu3bp1kV9DNecAgFKlLn6XJ9f/+uabbwJJMeSVLn3x\nLadbt25B35a82vPoo4/qWK2bKq/ykze5L9euXdvr8Xnz5ulYvQc73TXXXKNj1TivSpUqekxeAX7o\noYeCmsuzzz6r4/r16+tYrQHs9M89srHgyy+/rOM6dep4/ay88vnLL78EN7EwIY8Jcs3uQMm1wOVn\nTrpINVaSxyJZfagaMAH5jQxlA7f09HQdmzxG8MomERERERERWY4nm0RERERERGS5El9G+/rrr+u4\nsDXYVPkIAJw8eVLHwboUL9cpVHlu2rQpKNsy7fPPP9exumwv5/TXX38t8mvJhhxlypSxIDsqisaN\nGwPILxsE8ku/yNtbb72lY7mOZnHccccdXvHevXv12F133aVjWeYZLm655RYAwI033qjH5DHeSqqB\nC+BZvn/llVcCYBmtL7JZ0zPPPHPZn5Ul+W63O2g5lSRyfVhZrqa88MILQc+hadOmADxvT5HN3px8\njJelm2+//baOZTMrX/vixIkTdaxuKynOZ5hwIUszVWmsLLuUa7eePXtWx8ePHwfgeUyVnzu++OIL\nHW/duhUA8J///EePZWRk6Fg2GAvnY7RscCVvhVKfK+TvqjBt2rTR8YULF3T8448/6njdunUAPEui\nz507V4yML49XNomIiIiIiMhyPNkkIiIiIiIiy5X4MlrZgbZZs2Y63r59u45jY2MBFFzi0rZtWx3v\n27cPgO+OZZdSl5uPHDmix2SXVik7OxuAc8poJVkGWFRPPPGEjn2tlSRLKGRM1nnyyScBeP7+nLh/\nBmr58uUA8rvHFpfsbijXl61Xrx4Az06RGzdu1HFERIRf2ws1shxIdTDdtWuXHhs7dmxQttu7d++g\nvK6TxcfH61iu0yip98UVK1YYycluVatW1fGdd97p9fiQIUN0LD8rWEmVzgLA6tWrvR6XZbTydhen\nefzxx3UsuwAXRt6+0LVrVwCeHWxlma2VpYOhoKByV7UWuq911oH8NdOB/M/ecj3ZunXr6ljecubv\nLSpOps5t1HqlgOc+K7spK/v379fx2rVrdfzTTz/pWH0GlLfsyO768t+Q6hAvu+fLLraB4pVNIiIi\nIiIishxPNomIiIiIiMhyJb6M9ssvv/QZS7JDliI7ESYkJOhYXU5u1apVodtWC1VnZWXpMVm+Ky9B\ny7KwcNajRw8Anl35rrjiCh0fPnwYAPD000/rsdzcXEPZOZ/s2HzDDTcA8Nx/w7m7m/TnP/9Zx9df\nfz0Az/Kewkp9ZHmJLD1SXfkAoEOHDgAK7ur5t7/9TcdTp04tStohSS4+r0q2VCkb4Fl6HCh5TJa/\nY5ZuFY2vMtFLyf09HLz55ps6vvvuu3WsPkt89NFHQc/hpptu0nG1atUAAO+//74e++CDD4Keg53U\nLQmDBg3y+fgPP/yg40OHDgEAOnbs6PNnK1asCMCzJHfu3Lk6PnjwYGDJhgD5mezDDz/UsSqdBfJv\nb/BVtn0pWT6rqFvLyLfp06frWJUqF9RhVp77/Pe//wUA/P3vf9dj6lzlUu3atQPg+Vnj3Xff1bE8\nN1L/biZPnqzHPvnkEx0HeosAr2wSERERERGR5Ur8lU1/HTt2TMdr1qzxerygq6S+yG975RVT9Q0D\n4Oy1rYpDXU2T35xJap6+/vprYzmFE3k1RwlW04pQI6/6zp8/X8eFrVclGyypb/qef/55PVbQlXn1\nvPvvv1+PRUdH61iuL1muXDkAwKRJk/TY+fPnL5tXSdanTx8dq8YDALBz504AwWtUJa8iy6uZaWlp\nOv7tt9+Csm0naN++vc9x2TSlsPU3nUau2yj3qZ9//hmAtQ1lIiMjdSyvXDzwwANe+QwePNiy7ZZ0\n6grMVVddpcdkUxT5vqeOpf369dNjci4bNmwIAKhevboeW7x4sY5vu+02HTttLc6oqCgAnpVlqhoN\nAI4eParjN954AwArzwKl9kcgv2EPAAwdOlTHLpcLgOdnNVntNG7cOB0XpzpNrT8rmxGOGTNGx7Iq\nVFUPBAuvbBIREREREZHleLJJRERERERElnNsGa0V1PpaU6ZM0WNyLT7ZBMdp5RbF8dlnn+m4c+fO\nXo/Pnj1bx7JZCFlPrpOnyHLNcFa6dP7hrrDSWVnmnZKSomNZZlQYVUb7yiuv6LHx48fr+Morr9Sx\n+h0tWbJEj4Vy07G+ffvqWP7/lMdSK6kS6QEDBuixP/74Q8cvvfSSjkO5PDlYVCMJ9felZOnWli1b\njORU0nXv3h2AZ8MkWaJdnKZfqgy0oPXBpY8//rg4aTpC2bJlAXiWNL/11ls+f1Y1S3nvvff0mDwe\nNWjQwOs5slTUyets3n777QCAp556So/JRj6yEZVsdkf+k/+m5frzqnQWyF8zU96yJ9fkLowsk61T\np46O1WdvtZY44HkroKTymTNnjh6z8pYTXtkkIiIiIiIiy/Fkk4iIiIiIiCzHMtrLePDBBwF4dpCU\nXW5//PFH4zmVFDVq1NCxLL1S5S6y3FCWsFm5ph5dJMut5DpkGRkZAIB///vfxnMKRbJDquz0WJzS\nWV9kaaws8yzKWr+hRK1fBxRcAhis9URVx19ZHi3XRPbVkZzyFbYvOnkd2MKkpqbq+JZbbtFxzZo1\nAXh28JWlcb169SryNtTzZJmotHv3bh3LzqrhQnaWVVQZM+B5K48vqkt+QTZs2KBjJ39G8VUmrz4n\nAEBOTo7JdMKCLHGVt3ZIFy5cAAC0adNGj8mO7o0bN/Z6zpkzZ3QcGxvrM1afXdTavJej1tkM1i0n\nvLJJREREREREluPJJhEREREREVmOZbSX+NOf/qRj2bFLUd28AGDr1q1GciqJ1OL2QP7CsdIHH3yg\n41DuqhkKOnbsqOMqVaroWC3Yq7rzUT7ZVVqRJSxWkqV1cru+cpALLg8cODAo+QSLKqEHgFq1aul4\n3rx5Qd+2WqhdCufjc3H5KjP0t7Oq02zevFnHzZo103FCQgIAoGvXrnpMdpuUC7TPmjXrsttQHSC/\n//57n49/++23Og7H91N1DJGlybL0W5YZqo7sycnJekx24FT7tRwbNmyYjmU3zm3btgWce0kiSzMV\nuf/+85//1PHixYsBsPt0oL766isdy9s55Oe2unXrAgAmTJigxwoqqVeluLI8tyC+ymfz8vJ0/Omn\nn+r44YcfBgAcOHCg0Nf1B69sEhERERERkeVcBZ09W7YBlyu4G7DYyy+/rOOnn34aAPDll1/qsW7d\nuuk4WOu1ud1uV+E/5S3Ycy2/VVy4cKGOy5Qpo+O0tDQAQO/evfVYSb3h3t95BkrWfv3RRx/pWK7T\npGL57ZVdSsI+/cYbb+h45MiRXo/L/dhKDz30kI7lOpvyyqb6tlF+Q+/vFQy75joyMlLHa9eu1bGc\nV9VgxYp1idU6yIDvb2PVN7UAMHny5IC350tJ2K/9lZSUpGO1rqzcJ9U6sUD+OqZ2CuW5Loxa+3Hn\nzp16TF5R6tKli47lFdNgKInvi6piR86PbEgmq0d8faZdvXq1jlXjx6VLl+qx6667TsczZ87U8V//\n+tdA0i6U6X1azY28ulUQ9TPTpk3TY7KRkroaB+T/XjIzM32+VtOmTXW8fv16AOabEZW040elSpV0\nrKooZWXlL7/8omO5FqqqIGrevLkea926dZG3K3+fstmYlWtq+pprXtkkIiIiIiIiy/Fkk4iIiIiI\niCzHBkHwLP+SN0ufO3cOgOdN08EqnS3JVAMgecm9oJJDVfpTUktnnaR69eoAgJtuukmPybVfS0L5\nbEnSs2fPoG9DrsnbpEkTAEVbF0+VxoXy8UWu+yVLgGVp97JlywB4lhMXJi4uTseq3BDwLO30VTpX\nlFKxcCYbu/lqVsX1ec157rnnAHjux6NHj9ZxsEtnSzpVdv+Xv/xFj3388cc6liW1ysSJE3Us51I1\nzFu0aJEek80gZcmyajzmlKZM6laSRx99tNCfVceEBx54QI/J2F9qX1a3XAFASkpKwK8bamTZqq9m\npIWZPXu2jgsqoz158iQAz9/3+++/r+OC1v0MBl7ZJCIiIiIiIsvxZJOIiIiIiIgsxzJaeK6NlZiY\nqGO1TqFc4yocPfbYYwA817WSPvvsMx3LkmMKrvvuuw+AZ1fOFStW2JQNAcAzzzyjY9X1sCB79uzR\n8b333gvAs+tcKJPHAdkpsnv37gCKt/bm0aNHdSzLDK+55prLPk+WC5E3X2vuydKu6dOnm0wn7PTt\n21fH99xzD4D8sjfAsxslXSS7ysr9t3///jpW+7AqTQZ8rzX94osv6jg2NlbHsuu+eg11fA51qlxz\nwYIFeuzDDz/UcenS+acEderUAeC7xD4Q6lYT+ft79tlndfzSSy9Zuj2nefLJJwEUrfRYdVM2sdZ1\nYXhlk4iIiIiIiCzHk00iIiIiIiKyXNiW0apyLgD4xz/+oeMTJ07o+IUXXjCaU0lVWOeyESNG6Jhd\naM2pV6+e19ixY8dsyCS8LV++XMfXX399kZ+3bds2Ha9bt87SnOy2Y8cOHcsOkgkJCQCARo0aFfm1\nZNdJadasWToeMGCA1+OyOy5dVLt2bR3L0kNFLrS+adMmIzmFq9tuu81rbOnSpTr+7rvvTKYTcmRJ\nrYyLSh4fZFmpLKO95ZZbAABVqlTRY6ozbihS3Uflv+2YmBifP3vrrbcC8Fx5YMyYMTou6LaqopK3\nV7Rs2TKg13K6oUOH6liVHMuSZykzM1PHsuOy3Xhlk4iIiIiIiCwXdlc21dpiEyZM0GMRERE6llcp\nNmzYYC6xECa/9SvqOoHHjx/3+Rz5LZqvtbMqVaqk48KuuMo1hNQ6W7m5uUXKLxT06NHDa+zzzz+3\nIZPQIL9J9dX0wNeVBgCYMWOGjmvWrOn1uHyt4qztaGLdz5JGrcOr/g7E7t27L/u4XJ9z69atAW/P\nCdq1a6djX/8GZLM3Ci55vDl9+jQA4M0337QrnbC2cOFCHcsrm3fddRcAz+qtcKl4+/LLL73GVGUK\n4Hll88KFCwCA9957T4/NnDlTx4888oiOfVVUkDe5dqY8LkRFRXn9rKwoVE2BAODs2bNByq74eGWT\niIiIiIiILMeTTSIiIiIiIrJcWJTRyjJZtXZm/fr19diuXbt0LJsFUdH88MMPxX7ORx99pOMDBw7o\nuFq1ajpWJSxWOHjwIADg5Zdftuw17ZCUlKTj6tWr25hJ6Jk6daqOX3/9da/HZXOOgsphCyuTLezx\nadOmXfZxKjpZFi1jhaWz3tRtJJdSa5mmpqaaTCfsyBI3+V53+PBhAGwKZBd53JbvDb179wbguW7w\n/PnzdZyVlWUgu5Ljiy++0LH8LKWa1QwbNkyPySZwN99882VfVzYmo4vkbTZXXXWV1+Oq9B7wLP1O\nT08PbmJ+4pVNIiIiIiIishxPNomIiIiIiMhyYVFG27BhQx37Ws9HdjWVJbV0kerQq0pKrNC3b98i\n/6zqdAb4LlNcsmSJjgtaG27t2rXFyK7kSk5O1rEqD8/IyNBj33zzjfGcQoVcc+qJJ57QcXR0tGXb\nOHLkiI63b98OALj//vv1mCwZp8C43W6fMRWsS5cuPsezs7MBeHYJJ+vJMlq5zy5btszrZ2XpXOXK\nlXWsflcUHLJT9nPPPQcAGDdunB4bO3asjgcOHAggfNb0Ve9pgGcHX7mWsqLWKL2UWiVA7vNPPfWU\nVSmGNPlv/sknn7zsz86dO1fHaWlpwUrJMryySURERERERJbjySYRERERERFZzrFltPXq1dOx7KCl\nyDI62YWSvN1xxx0APC/rlylT5rLPadq0qY4L6yr77rvv6njPnj1ej3/yySc63rFjx2Vfy4muvPJK\nHXfr1s3r8Y8//ljHqkSFvO3du1fHKSkpOr799tsBACNHjgx4G7JD3+TJkwN+PSpYuXLlvMbCpZyt\nOOSxWt5SIv3+++8AgPPnzxvJiTyp4/aAAQP02KhRo3ScmZmp43vvvddcYmFu9uzZAIDhw4frMfV5\nCABeeOEFAP515A9F8vj6yCOP6DgqKgoAcMMNN+ixqlWr6lh+rpszZw4AYMyYMUHKMvSo+du2bZse\nK+gzttrX5PyHAl7ZJCIiIiIiIsu5gt1YweVy2dK5QV5hePrpp70eb926tY4LaipjF7fb7b1oXBHY\nNdehyt95BszOtfyG6+uvv9axWputf//+eiw3N9dUWsUSCvt0165ddSyb+sj1rlQzqhkzZugxucaj\n/GbSrkYeoTDXVlBr5wL567y9+OKLeszEmpGhMNdynel//etfOr7vvvt0rK7glOSrZqEw14WRzWfi\n4+N1rI4h8vPYO++8o2O5X+/bty+YKYbM+6JJdevW1bG8Sjdv3jwAnleki8MJ+7SkGiYBQNu2bXX8\n/PPP61h9bjGtJM+1Widz8eLFeqygc7Nbb70VALBmzZpgp+U3X3PNK5tERERERERkOZ5sEhERERER\nkeUcVUablJSkY7U2JJB/863EMlpiuZA53KfNCZe5/vzzz3U8fvx4AOZLi0JtrmvWrKnjl156Sceb\nN28GULKbWoXaXPsiP6Oo5jJA/vrIU6dO1WPHjh3T8blz5wxkdxHfFy9PNpy88cYbAQBt2rTRY/JW\nisI4YZ8OFSV5rr///nsAnqX1klzndfTo0cFOJ2AsoyUiIiIiIiIjeLJJRERERERElnPUOps33XST\njn2VzgLArl27AACnTp0ykhMREVlPdgmmovn55591PHjwYBszCU/r1q3TcYcOHWzMhPzVp08fHavy\nx0aNGumx4pTREgFAlSpVAHh2tpdde99++23jOVmNVzaJiIiIiIjIcjzZJCIiIiIiIss5qoy2IKrU\nAchfEPXXX3+1Kx0iIiIiCjEnTpzQcf369W3MhJxCdVNXfwPAiy++qOMDBw4Yz8lqvLJJRERERERE\nlnPUOptOUZLXA3ISridmDvdpczjX5nCuzeFcm8H3RXO4T5vDuTaH62wSERERERGRETzZJCIiIiIi\nIssFpUFQZGTkwd9//72aP88tV67coTNnzlS3Oien8neuOc/Fx7k2h3NtBo/V5nCuzeHxwxzOtTmc\nazN4rLZeUO7ZdLlcbn9f1+VyBXTPQLjxd645z8XHuTaHc20Gj9XmcK7N4fHDHM61OZxrM3isth7L\naImIiIiIiMhyPNkkIiIiIiIiy/Fkk4iIiIiIiCxn7GQz65csJE5PRIvpLZA4PREVX62ICf+ZYGrz\nYSXnRA46zOqAplOaIn5qPOc5iIYsHoJqb1RDs6nN7E7F8VbuXInGkxojZmIMXlv3mt3pOF6eOw8t\nprdAr3m97E7FsXj8MIv7tBmpG1IRPzWenz8MuPbta9F8WnMkTk9E65mt7U7H0bhf+8/YyWbM1THI\nGJ6B74Z/h833b0b5MuWR3DjZ1ObDSulSpTG+y3hkPpCJ9UPWY/L/m4wdR3fYnZYjDUochFV3r7I7\nDcfLc+dhxPIRWHX3KmQ+kIl5W+dxnw6y1A2paBLdxO40HI3HD7O4Twdf5uFMvJPxDjYN24Qtw7dg\nadZS7D622+60HKuUqxTS7k1DxvAMbBy20e50HIv7dWBsKaNdvXs1GlZpiDoV69ixecerHlUdCdUT\nAABRV0Qh9ppY7D+x3+asnCmpbhIql6tsdxqOt3H/Rlx39XWoV6keykSUQUpcChbvWGx3Wo6VcyIH\ny3cux9AWQ+1OxdF4/DCH+7QZ249uR5tabVC2dFlElIpA+3rtsWj7IrvTciw33Mhz59mdhuNxvw6M\nLSebC7YSi1lCAAADOElEQVQuQL+4fnZsOuzs+W0Pthzcgja129idCpHf9p/YjzoV8r+cql2hNvaf\n5BcowTJq1SiM6zQOLrCDOzkD92kz4qrGYW32Whw7cwy553Ox/P8vx77j++xOy7FccKHTnE5oNbMV\nZm6eaXc6jsX9OjClTW/w/B/nsSRrCV7t+KrpTYedU+dOoc/CPkjtmoqoK6LsToeIQsCyrGWoVr4a\nEqonIG1PGtywfi1mIpO4T5vT+JrGGP2n0eg0pxOirohCYvVERJSKsDstx0ofnI4aV9XAkdNH0GlO\nJ8RGxyKpbpLdaTkO9+vAGL+yuWLnCrSs0RLR5aNNbzqsXMi7gD4L+2Bgs4Ho3bi33ekQBaRWhVrI\nPp6t/zvnRA5qXVXLxoycK31fOpb8uAQNUhug3yf9sOanNbjn03vsTovIb9ynzRqUOAib7t+EtPvS\nUKlcJcRcHWN3So5V46oaAIDo8tFIbpyMjft532awcL/2n/GTzXlb57GE1oDBiwejSXQTjGw70u5U\nHM/9v/9R8LSq2Qo7f92Jvb/txbk/zmH+1vnodT07SgbD2FvHIntUNnaP3I35d85Hh/odMDt5tt1p\nORaPH8HHfdqsI6ePAACyj2fj0x2fon98f5szcqbc87k4de4UAOD0udP4YvcXiKsaZ3NWzsX92n9G\ny2hzz+di9e7VmNFjhsnNhp307HTM/e9cxFeNR+L0RLjgwthbx6Jro652p+Y4/T/pj7Q9afjlzC+o\n+1ZdPH/z8xiUOMjutBwnolQEJnWbhM4fdEaeOw9DEocgNjrW7rSIAsLjBznRnQvvxK9nfkWZiDKY\n0n0KKpStYHdKjnTo1CEkL0iGy+XChbwLGBA/AJ0bdrY7Lcfifu0/l9tt/TeqLpfL7e/rulwuuN1u\n3sFfRP7ONee5+DjX5nCuzeCx2hzOtTk8fpjDuTaHc20Gj9XWs6UbLRERERERETkbTzaJiIiIiIjI\ncjzZJCIiIiIiIsvxZJOIiIiIiIgsF5RutOXKlTvkcrmq+ftcq/NxMn/nmvNcfJxrczjXZvBYbQ7n\n2hweP8zhXJvDuTaDx2rrBaUbLREREREREYU3ltESERERERGR5f4PQ+8P2ZkutsAAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x27815670>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def make_image(X):\n",
    "    im = np.swapaxes(X.T, 0, 1)\n",
    "    im = im - im.min()\n",
    "    im = im * 1.0 / im.max()\n",
    "    return im\n",
    "\n",
    "plt.figure(figsize=(16, 5))\n",
    "for i in range(0, 10):\n",
    "    plt.subplot(1, 10, i+1)\n",
    "    plt.imshow(make_image(test_data[i][0]), interpolation='nearest', cmap=plt.get_cmap('gray'))\n",
    "    true = test_label[0][i]\n",
    "    pred = FPGA_predicted[i]\n",
    "    color = 'green' if true == pred else 'red'\n",
    "    plt.text(0, 0, true, color='black', bbox=dict(facecolor='white', alpha=1))\n",
    "    plt.text(0, 32, pred, color=color, bbox=dict(facecolor='white', alpha=1))\n",
    "\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Save our model\n",
    "Let's save the weights in pickle format, so we don't need Caffe next time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "values = lasagne.layers.get_all_param_values(net['output'])\n",
    "pickle.dump(values, open('model.pkl', 'w'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
